We’ve open-sourced Klarity - a tool for analyzing uncertainty and decision-making in LLM token generation. It provides structured insights into how models choose tokens and where they show uncertainty.
What Klarity does:
<ul>
<li>Real-time analysis of model uncertainty during generation</li>
<li>Dual analysis combining log probabilities and semantic understanding</li>
<li>Structured JSON output with actionable insights</li>
<li>Fully self-hostable with customizable analysis models</li>
</ul>
The tool works by analyzing each step of text generation and returns a structured JSON:
<ul>
<li>uncertainty_points: array of {step, entropy, options[], type}</li>
<li>high_confidence: array of {step, probability, token, context}</li>
<li>risk_areas: array of {type, steps[], motivation}</li>
<li>suggestions: array of {issue, improvement}</li>
</ul>
Currently supports hugging face transformers (more frameworks coming), we tested extensively with Qwen2.5 (0.5B-7B) models, but should work with most HF LLMs.
Installation is simple: <code>pip install git+https://github.com/klara-research/klarity.git</code>
We are building OS interpretability/explainability tools to visualize &amp; analyse attention maps, saliency maps etc. and we want to understand your pain points with LLM behaviors. What insights would actually help you debug these black box systems?
Links:
<ul>
<li>Repo: <a href="https://github.com/klara-research/klarity" target="_blank" rel="noopener nofollow">https://github.com/klara-research/klarity</a> - Our website: <a href="https://klaralabs.com/" target="_blank" rel="noopener nofollow">https://klaralabs.com</a></li>
</ul>

Squad for devs building apps with GenAI and LLMs! Connect, share, exchange ideas, and get feedback. 

Build With GenAI

Klarity is an open-source tool designed to analyze uncertainty and decision-making in LLM token generation. It provides real-time analysis, combining log probabilities with semantic understanding, and generates structured JSON outputs. The tool supports Hugging Face transformers and is customizable for different models.

Klarity – Open-source tool to analyze uncertainty/entropy in LLM output