Replace 100 Questions?
KINDA · weekend projectA personal CLI that asks the same questions across four model APIs and compares the answers is weekend-buildable, but matching the product's web-grounded runs, source normalization, failure handling, durable evidence, scoring, and polished reports takes substantially more work.
Build me a local AI visibility benchmark for one brand. Requirements: - Use Node 22, TypeScript, official provider SDKs, SQLite, and a CLI. - `benchmark --domain example.com --description "..."` creates one immutable run. - Generate 25 buyer questions from the domain and description, or accept a JSON question file. - Ask the exact same questions through OpenAI, Anthropic, Gemini, and xAI. - Use each provider's supported web-search or grounding tool; keys live only in `.env`. - Limit concurrency per provider, retry transient failures, and preserve failed cells in the report. - Store prompts, raw answers, citations, timestamps, model ids, and errors in SQLite. - Detect exact and case-insensitive brand mentions; allow aliases in a config file. - Extract named competitors with one structured LLM pass after all answers are stored. - Normalize citation URLs by hostname, canonical URL, and stripped tracking parameters. - Compute visibility by provider, answer coverage, owned-domain citation rate, and top sources. - Show missed questions where competitors appear but the target brand does not. - Render a self-contained static HTML report with filters and expandable raw evidence. - Export questions, answer metrics, competitors, and citations as CSV files. - Every aggregate metric must link back to the answer rows used to calculate it. - Out of scope: accounts, billing, teams, scheduled monitoring, and recommendation generation. - Include fixture-based tests for mention detection, URL normalization, and metric calculations. - README: setup, provider-specific grounding caveats, estimated API cost, and exact run commands.
$ open in your agent (prompt prefilled, you press enter) or copy it raw
They pay for a repeatable, frozen benchmark with provider failures handled, citations normalized, every metric tied to evidence, and a report that is ready to act on.
xreliable orchestration and retries across four providers
xnormalized citations and evidence-linked metrics
xcompetitor and missed-question extraction
xstored point-in-time reports and comparisons
xpolished exports and action recommendations
No mature prior art is listed for this record.
What does the 100 Questions verdict mean?
The core job looks buildable, with meaningful gaps: reliable orchestration and retries across four providers, normalized citations and evidence-linked metrics. Read the full tradeoff list before committing. This research record is not a hosted IVCIFY tool.
What price does this directory record show for 100 Questions?
The directory records $9/month for First benchmark, checked 2026-07-31. Verify the source before making a purchase decision. This reference stays outside retail Stack Math unless current matched evidence supports the comparison.
What do I lose by replacing 100 Questions?
Honestly: reliable orchestration and retries across four providers; normalized citations and evidence-linked metrics; competitor and missed-question extraction; stored point-in-time reports and comparisons; polished exports and action recommendations. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to 100 Questions?
This record does not list mature open-source prior art. The build prompt remains available as a research starting point.