No learning during training

[oldcask]

Hi @gsurma,

Thanks for sharing the code, great work there.

I was try to recreate the Breakout model by running the training step. However, even after 7000 runs there seems to be no learning happening. The average score is constant between 1-1.5. However, the accuracy increases and loss decreases.

The code I am using is almost exact clone of the current repo. Could you please let me know if there was any update that was done by you before running the training step for the game?

Tried the same for Pong game and failed to see any learning happening.

Happy to share more details, and any help will be appreciated. Thank you!

[gsurma]

Hi,

Can you share your hyperparams and loss plots? It's hard to debug your issue without them.

[oldcask]

I trying to reproduce similar results to what you have shared. Below is the hyperparameters and loss plots.

Hyperparameters

• GAMMA = 0.99
• MEMORY_SIZE = 900000
• BATCH_SIZE = 32
• TRAINING_FREQUENCY = 4
• TARGET_NETWORK_UPDATE_FREQUENCY = 40000
• MODEL_PERSISTENCE_UPDATE_FREQUENCY = 10000
• REPLAY_START_SIZE = 50000
• EXPLORATION_MAX = 1.0
• EXPLORATION_MIN = 0.1
• EXPLORATION_TEST = 0.02
• EXPLORATION_STEPS = 850000

Also, should the ClippedRewardsWrapper(env) be used to allow clipping of rewards? It seems to commented in your code.

Thanks.