New Way to Watch LLM Work Locally with Langfuse

Running your Large Language Models locally with Langfuse is now easier. This new method gives developers more control over their data compared to cloud services.
By Newsroom | Technology, Artificial Intelligence |

A new approach to monitoring Large Language Model (LLM) operations, centering on self-hosted 'Langfuse', is gaining traction. This method promises greater control and deeper insights by running the entire observability stack locally. The core setup involves integrating 'Langfuse' with 'vLLM', a high-performance LLM inference service.

The setup utilizes Docker to deploy the necessary components: the 'Langfuse Server' for data ingestion, a 'Langfuse Worker' to process events and power dashboards, a 'PostgreSQL' database for raw trace data storage, and 'vLLM' itself for model inference. This interconnected system allows developers to send trace information from their LLM applications directly to the local 'Langfuse Server'. From there, the data is persisted in 'PostgreSQL' and then visualized in real-time, offering a continuous stream of operational awareness.

Technical Underpinnings and Integration

The 'Langfuse' system is designed for flexibility. Its open-source nature allows for self-hosting, mirroring the infrastructure that powers its cloud-based counterpart. Installation is typically managed via 'Docker Compose', simplifying the deployment of distinct application containers. These include:

Read More: Bright Vision Technologies hiring AI engineers in May 2026

  • Langfuse Web: Serves the user interface and application programming interfaces.

  • Langfuse Worker: Handles asynchronous event processing.

The operational flow begins with the 'Langfuse SDK' within an application script. This SDK dispatches trace information to the 'Langfuse Server'. The server then routes this raw data to the 'PostgreSQL' database. The entire process culminates in real-time data visualization, making operational patterns and potential issues readily apparent.

Customization and Application Examples

For developers looking to integrate this stack, specific configurations are available. The 'Langfuse' project uses a 'tagged semver' release policy for version management. Installation typically involves cloning the 'Langfuse' repository and adjusting the 'docker-compose.yml' file to include sensitive credentials and environment variables, such as 'LANGFUSEPUBLICKEY', 'LANGFUSESECRETKEY', and 'LANGFUSE_HOST'.

The integration extends to popular LLM development frameworks. For instance, 'LangGraph' can be wired with 'Langfuse' callbacks for tracing. An example demonstrates using 'vLLM' as the backend for a 'LangGraph' chatbot, specifying the model to be served and its local endpoint. This involves installing requisite libraries like 'langfuse', 'langchain', 'langgraph', and 'langchain_openai'. The configuration includes setting environment variables for 'Langfuse' and potentially an 'OpenAI API Key', even when using a local model served via 'vLLM'.

Read More: Why AI Models Give Wrong Answers on 19 May 2026

The process highlights how to connect 'LangGraph' applications with 'Langfuse' and 'vLLM', detailing the setup for both the LLM inference backend and the observability layer. This comprehensive approach is geared towards providing a robust, self-managed solution for understanding and optimizing LLM deployments.

Frequently Asked Questions

Q: What is the new way to watch Large Language Models work?
A new way to watch LLM work is by using self-hosted 'Langfuse'. This means running the whole system on your own computers for more control.
Q: How does self-hosted Langfuse work with LLMs?
It works by connecting 'Langfuse' with 'vLLM', which is a service for running LLMs. You use Docker to set up 'Langfuse Server', a 'Langfuse Worker', a 'PostgreSQL' database, and 'vLLM' to see how your LLM is working.
Q: Can developers customize the Langfuse setup?
Yes, developers can customize it. They can clone the 'Langfuse' project and change the 'docker-compose.yml' file to add their own keys and settings.
Q: How can Langfuse be used with other LLM tools like LangGraph?
'Langfuse' can be connected to tools like 'LangGraph' using special callbacks. This lets you see the steps your 'LangGraph' chatbot takes when it uses 'vLLM' for its answers.