What is rl_for_game?

This project is a Python toolkit that trains an AI to play games on your computer by watching the screen and sending keyboard and mouse inputs. You record footage of yourself playing, annotate which actions you performed, and then feed that data to a neural network that learns to mimic and improve on your behavior. Once trained, the AI can take over your screen in real time, with F5 to let it play and F6 to take back control. The core learning algorithm is called Dueling DQN, a variant of a technique called Q-learning that estimates how good each possible action is at each moment. On top of that, the project layers a Transformer, a type of neural network good at processing sequences, so the AI can reason across multiple frames rather than just reacting to a single snapshot. This combination helps the agent handle situations where what happened a few seconds ago matters as much as the current frame. Several practical features are built in to make training more stable. A Double DQN mode prevents the AI from overestimating how good its choices are. An action-balanced sampler corrects for situations where one action (like doing nothing) shows up far more often than others in training data, which would otherwise skew the model. You can spread training across multiple GPUs without changing your config, and you can preprocess frames into a compact format to speed up disk reading during training. For data collection, the toolkit includes a screen recorder with an annotation dialog. You record a session, mark key moments with a keyboard shortcut, and then select which actions and rewards applied to those moments. All settings, from learning rate to screen resolution to how many Transformer layers to use, live in a single YAML file, and command-line flags override any setting without editing that file. There is also an online learning mode where the AI plays and trains at the same time, updating its own reward model and replay buffer on the fly. This requires a pretrained reward model to start from. A TensorBoard log tracks all training and online metrics so you can watch the AI improve over time.