Home
Artists
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register
Partychan Telegram ThePornDude
Home Artists Posts Import Register

Langchain: Auto evaluate llms and agents video files: (Patreon)

Published:
2023-02-26 00:19:25
Edited:
2023-12-02 04:45:33
Imported:
Tags:

Downloads

Content

files are for video: https://youtu.be/6ukUopbOr58


Langchain evaluation docs: https://langchain.readthedocs.io/en/latest/use_cases/evaluation.html miniconda: https://docs.conda.io/en/latest/miniconda.html Huggingface datasets: https://huggingface.co/docs/datasets/installation


echohive blog: https://echohive.ghost.io/

Comments

Mark

This is amazing, would this work with unstructured data?

echohive42
>>102821704

Yeah it should. All this is doing is evaluating the return from an llm and ground truth by getting an llm to compare the two.

Brian Porter

I'm a bit late to the game on this one but I'm running into parsing issues when trying to run the agent_eval.py file. I will get a few observations and actions but then will hit an error like: "langchain.schema.OutputParserException: Could not parse LLM output: `I need to read the article to find the exact percentage. Action: Click on the article from the previous search`" Some fixes online suggest writing a custom parser but that seems pretty complicated (I am a novice). Anybody else run into this issue and find a way to resolve it?