AgentBench: Evaluating LLMs as Agents

Evaluating Large Language Models (LLMs) as agents in interactive environments, highlighting the performance gap between API-based and open-source models, and introducing the AgentBench benchmark.

arxiv.org

Saved by Darren LI

RelatedCollectionsHighlightsNotes

LLM Powered Autonomous Agents

Lilian Weng lilianweng.github.io

memary: Open-Source Longterm Memory for Autonomous Agents

memary demo

Why use memary?

Agents use LLMs that are currently constrained to finite context windows. memary overcomes this limitation by allowing your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve relevan... See more

GitHub - kingjulio8238/memary: Longterm Memory for Autonomous Agents.

Thumbnail of LLM Powered Autonomous Agents

Lilian Weng • LLM Powered Autonomous Agents

Large variety of ready-to-use LLM evaluation metrics (all with explanations) powered by

ANY

LLM of your choice, statistical methods, or NLP models that runs

locally on your machine

:

G-Eval

Summarization

Answer Relevancy

Faithfulness

Contextual Recall

Contextual Precision

RAGAS

Hallucination

Toxicity

Bias

etc.

LLM Powered Autonomous Agents

GitHub - kingjulio8238/memary: Longterm Memory for Autonomous Agents.

Lilian Weng • LLM Powered Autonomous Agents

GitHub - confident-ai/deepeval: The LLM Evaluation Framework