What is sky-with-you?

This project lets an AI companion play as a physical presence inside Sky: Children of the Light, a PC game. It watches the game's chat panel, replies to messages, lights and receives candles, bows, accepts teleport invitations, holds your hand when you reach out, and can say a line like come on, let's go home before guiding you back to your home area. The system runs entirely on the same Windows machine as the game and is built in three layers. A perception layer takes screenshots and combines visual matching with template recognition and text reading to build a shared picture of what is happening on screen, including which panels are open, whether a popup is showing, and what scene the game is in. A scheduling layer reads messages, asks a language model to write a reply based on a persona file the user sets up, and decides what action to take next, always working from that shared picture rather than the live screen. An execution layer sends the actual keyboard presses using an external Arduino board that plugs in over USB, so the game reads them as real hardware key presses instead of software simulated ones, since the game blocks software key simulation. This layer also handles Chinese text input through the clipboard, takes screenshots, and checks that the game window is in focus before pressing anything. Setting it up requires a Windows PC running the game, a small Arduino compatible board flashed with the included firmware, and an OpenRouter API key for the language model. You edit a persona file to describe who your AI companion is, then run two Python scripts in separate terminals. The project also documents several safety details worked out through trial and error, such as refusing to send key presses when the game window is not focused, and ignoring the automation's own messages when they get read back by the text recognition step so it does not talk to itself. The README notes this only reads the screen and sends keyboard input. It does not modify the game or read its memory, and automating gameplay this way may violate the game's terms of service, so use is at your own risk. The project builds on an earlier keyboard injection tool credited in the README. It is released under the MIT license.