Skip to content

TensorFlow record files are corrupted #26

@griffbad

Description

@griffbad

I'm trying to train an FFN, and the first 2 steps (partition and build the coordinate file) seem to go fine, but training throws Key Value errors. On further inspection (using TFRecord Viewer), I get this error:
tensorflow.python.framework.errors_impl.DataLossError: corrupted record at 0.

Any help would be super appreciated!

TF version 1.13.2, and here are the exact calls I'm making:

For computing partitions:
python ../../ffn/compute_partitions.py \ --input_volume ../training_data_img.h5:label \ --output_volume training_data2.h5:af \ --thresholds 0.025,0.05,0.075,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9 \ --lom_radius 3,3,3 \ --min_size 10000

For building the TFRecord file:
python ../../ffn/build_coordinates.py \ --partition_volumes validation:training_data2.h5:af \ --coordinate_output tf_record_fileyw \ --margin 3,3,3

and for training:
python ../../ffn/train.py \ --train_coords tf_record_fileyw \ --data_volumes validation1:../training_data_img.h5:image \ --label_volumes validation1:../training_data_img.h5:label \ --model_name convstack_3d.ConvStack3DFFNModel \ --model_args "{\"depth\": 2, \"fov_size\": [2, 2, 2], \"deltas\": [2, 2, 2]}" \ --image_mean 72 \ --image_stddev 33

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions