LLMonitor

LLMonitor (now rebranded as Lunary) is an open-source observability and analytics platform for AI applications and agents. It enables developers to monitor LLM requests, track costs, tokens, and latency, debug complex agent executions, and trace user conversations to identify gaps in chatbot knowledge. The platform also supports capturing user feedback and creating labeled training datasets for model fine-tuning. It offers both a hosted cloud version and a self-hostable open-source option with built-in integrations for Python and JavaScript.