Nikhil C R
Web Research Agent
Searches, judges what it found, scrapes deeper when needed, and writes a report.
Problem
Open research questions need several searches, a judgement on whether search snippets go deep enough, and full-page reading when they don’t. The agent automates that loop: it works out what the query needs, picks web, news or both, escalates to scraping when snippets fall short, and says where its confidence is low.
Architecture
The planner fans out to web search, news search or both in parallel. An LLM evaluator then routes to scraping, straight to synthesis, or back to planning, capped at 8 research iterations.
- FastAPI serviceapi.py:34-48
POST /research runs the full graph and returns the report.
- React UIApp.js
Query box and a Markdown-rendered report.
- CLImain.py
Single query, interactive mode, optional news search, save to file.
- Search toolstools.py
Tavily for web, NewsAPI for news, Crawl4AI for full pages.
Key decisions
Reducers so parallel branches can merge
List fields use operator.add and dict fields use a custom dict_merge reducer, so the web and news branches can both write to shared state.
state.py:9-40The planner chooses the fan-out
The LLM returns a search approach; “parallel_search” becomes a list of two nodes, so both searches run at once. News search can be switched off.
agent.py:41-57Self-evaluation with a rule-based override
An LLM decides whether snippets are sufficient and picks up to 5 URLs. Code forces scraping for subjective queries with thin snippets, and scrapes the top 3 if the LLM asks for more but picks none.
nodes.py:375-404Loop-safe by design
Research iterations are capped at 8. Past the cap the agent still synthesizes, but flags the fallback and low confidence in the report.
agent.py:79-85
Tests
- tests/test_agent_edge_cases.pyview
Missing API key fails fast; graph exceptions surface as error-log entries; the planner routes to synthesis once the iteration cap is hit.
- tests/test_evaluate_results_and_select_urls.pyview
With the LLM mocked: no results or invalid JSON loop back to planning; sufficient snippets route to synthesis.
Run it
python main.py "Recent advancements in AI filmmaking"uvicorn api:app # then POST /research {"query": "..."}