Finalize transition to TFv2

[gpovysil]

I would love to try out the TFv2 version of DeepProfiler, however, I cannot even run the tests or BBBC021 example because not everything has been switched to the new version. I tried to fix the config file by changing the names of some of the parameters, but always get another error. I also played around with the test code but experienced similar issues.
Before I spend hours debugging the code, I wanted to ask if you have a rough timeline for the next release?
I can also give some examples about the issues I run into if that is helpful.
Thanks

[Arkkienkeli]

Hi, we did not support tests much in the latest releases, however the example data should work. Could you provide the error that you get? Also you might want to try current master branch.

[gpovysil]

Thanks for the quick reply. I did try both, the master and tf2 branch and tried debugging on the master. I used the example data from the wiki and changed training_values/validation_values to training/validation, queue_size to cache_size, cnn to efficientnet and added label_smoothing to the config.json (prompted by the errors I got). Then I get the following error:

Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/xxx/DeepProfiler/deepprofiler/main.py", line 216, in
cli(obj={})
File "/home/xxx/deepprofenv/lib/python3.8/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/home/xxx/deepprofenv/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/xxx/deepprofenv/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/xxx/deepprofenv/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/xxx/deepprofenv/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/xxx/deepprofenv/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/xxx/DeepProfiler/deepprofiler/main.py", line 172, in train
deepprofiler.learning.training.learn_model(context.obj["config"], dset, epoch, seed)
File "/home/xxx/DeepProfiler/deepprofiler/learning/training.py", line 41, in learn_model
model = model_module.model_factory(config, dset, crop_generator, val_crop_generator, True)
File "/home/xxx/DeepProfiler/plugins/models/efficientnet.py", line 21, in model_factory
return createModelClass(DeepProfilerModel, config, dset, crop_generator,
File "/home/xxx/DeepProfiler/plugins/models/efficientnet.py", line 142, in createModelClass
return ModelClass(config, dset, crop_generator, val_crop_generator, is_training)
File "/home/xxx/DeepProfiler/plugins/models/efficientnet.py", line 29, in init
self.feature_model, self.optimizer, self.loss = self.define_model(config, dset)
File "/home/xxx/CellPainting/DeepProfiler/plugins/models/efficientnet.py", line 91, in define_model
y = tf.compat.v1.keras.layers.Dense(self.config["num_classes"], activation="softmax", name="ClassProb")(features)
KeyError: 'num_classes'

[Arkkienkeli]

Could you try to switch crop_generator to sampled_crop_generator, export single-cells and then run training?
That is the workflow that we aim for now.
Sorry for inconveniences , we are now in transition period :/

[gpovysil]

No worries. It is working now, thanks a lot. Will try my own data next.