Github KoboldCPP

Github KoboldCPP is a free, open-source tool designed to run large language models (LLMs) locally on your computer with an integrated web UI. It supports text, image, video, speech, and music generation using a wide array of compatible open models. KoboldCPP is suitable for users who want to chat with AI, write stories, roleplay, or deploy local AI APIs for app integrations. No installation is required, and binaries are available for Windows, Linux, and macOS, making it accessible for hobbyists, creators, and developers.