Open knowledge sources for commercial agent platforms: access, licences and limits (Oct 2026)

Research date: 2026-10-22. Every example request below was run on that date with curl from a residential IP, using User-Agent: GAKN-Research/0.1 (https://gakn.online; aabscnonprofit@gmail.com). No accounts were created and no keys were used.

GAKN display model, assumed throughout: each external hit is shown as metadata + short snippet + link to the original, always attributed to its source and never presented as GAKN's own content. GAKN may charge for its own services (search, ranking, agent tooling, hosting of its own knowledge). It must not sell third-party content as such.

This is a research note, not legal advice. Items marked "owner decision" need a human call before connecting.

Summary table

#	Source	Key needed	Tested (HTTP / results)	Metadata license	Commercial OK	Verdict
1	Wikipedia (Action API + REST search), all languages	No (UA required)	200 / 3,860 hits (en); ru, de, ja also 200	Text CC BY-SA 4.0	Yes, with attribution + license link	CONNECT NOW
2	Wikidata (wbsearchentities, SPARQL)	No (UA required)	200 / 5; SPARQL 200 / 5	CC0	Yes	CONNECT NOW
3	Wikimedia Commons	No (UA required)	200 / 5 files	Per file (PD, CC BY, CC BY-SA...)	Per file license	CONNECT NOW
4	YouTube Data API v3	Yes (Google API key)	403 without key	YouTube API ToS	Restricted; no API reselling	DO NOT CONNECT
5	SepiaSearch (global PeerTube index)	No	200 / 72 (en), 2,122 (fr filter)	Per video licence; no API terms published	Metadata + link only	CONNECT NOW
6	PeerTube single instance (e.g. framatube.org)	No	200 / 1,254	Per instance / per video	Metadata + link only	CONNECT NOW (via Sepia; direct instances optional)
7	OpenAlex	Works keyless, but key strongly needed	200 / 664,957	CC0	Yes	CONNECT WITH KEY (free; owner action)
8	Crossref REST	No (mailto for polite pool)	200 / 44,266	Facts, "use for any purpose"; abstracts may be copyrighted	Yes	CONNECT NOW
9	arXiv API	No	200 / 437 (Atom XML)	Metadata CC0; e-prints copyrighted	Metadata yes	CONNECT NOW (1 req / 3 s)
10	Europe PMC REST	No	200 / 177,562	Per article; EMBL-EBI terms	Yes, respect third-party rights	CONNECT NOW
11	Semantic Scholar Graph API	Keyless pool is saturated	429 (3 attempts)	S2 API License; data under per-dataset licenses (some CC BY-NC)	Unclear; no repackaging	CONNECT WITH KEY (owner action + legal review; low priority)
12	Internet Archive advancedsearch	No	200 / 4,441	IA claims no new rights; items vary	ToU says "scholarship and research purposes"	DO NOT CONNECT (owner decision)
13	Open Library search	No	200 / 1,747	Not stated on API page	API "not intended" as backend for commercial services	DO NOT CONNECT (live); owner may ask OL or use dumps
14	Project Gutenberg via Gutendex	No	200 / 231 (en), 43 (fr)	PG catalog public domain	Yes	CONNECT NOW (low volume; self-host later)
15	Europeana Search API	Yes (free wskey)	401 without key; 200 / 409 with public demo key	Metadata CC0; items per rights	Yes	CONNECT WITH KEY (owner action)
16	Zenodo	No (for search)	200 / 3,476	Metadata CC0	Yes	CONNECT NOW
17	DOAJ	No	200 / 20,739	CC0	Yes	CONNECT NOW
18	data.gov catalog (new, non-CKAN)	No	200 / 20 per page (cursor)	US gov metadata; dataset licenses vary	Yes	CONNECT NOW (API is v0.1.0, expect changes)
19	Hacker News (Algolia)	No	200 / 2,925	No published terms; posts owned by authors	Titles + links only	CONNECT NOW
20	Stack Exchange API	Keyless: 300 req/day/IP	200 / 5 (quota 299/300)	CC BY-SA (content)	Yes, with attribution rules	CONNECT WITH KEY (free app key; owner action)
21	Openverse (CC media)	No (anonymous tier)	200 / 240	Per work CC license	ToS reserves right to charge commercial users	CONNECT WITH KEY (register + ask about commercial use)
22	Library of Congress loc.gov JSON	No	200 / 1,100	Per item rights	Per item	CONNECT NOW (20 req/min hard limit)
23	PubMed E-utilities (NCBI)	No (3 req/s); key gives 10 req/s	200 / 71,511	Citation data public; abstracts may be copyrighted	Yes	CONNECT NOW (optional; overlaps Europe PMC)
24	dblp	No	202 bot-challenge page (Anubis)	CC0	Yes	DO NOT CONNECT (live API blocked for bots; use dumps)

Caching rule of thumb used below: CC0 / public-domain metadata may be stored indefinitely (refresh for freshness). CC BY-SA text may be stored indefinitely if attribution travels with it. YouTube: max 30 days. Copyrighted abstracts: store if needed for ranking, but display only a short snippet and link out.

---

1. Wikipedia (MediaWiki Action API and REST search)

• Base endpoints (any language edition, replace en with de, ru, ja, ...):
  • Action API: https://{lang}.wikipedia.org/w/api.php
  • REST search: https://{lang}.wikipedia.org/w/rest.php/v1/search/page
  • Page summary: https://{lang}.wikipedia.org/api/rest_v1/page/summary/{title}
• Tested
  • https://en.wikipedia.org/w/rest.php/v1/search/page?q=photosynthesis&limit=5 -> 200, 5 pages
  • https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch=photosynthesis&srlimit=5&format=json&formatversion=2&srprop=snippet|timestamp|size -> 200, totalhits 3,860
  • https://ru.wikipedia.org/w/rest.php/v1/search/page?q=фотосинтез&limit=5 -> 200, 5 (first: Фотосинтез); de and ja also 200.
  • https://en.wikipedia.org/api/rest_v1/page/summary/Photosynthesis -> 200
• Auth: none. User-Agent required: descriptive, with contact info, format <client>/<version> (<contact>) <lib>/<version>; automated agents should include "bot"; generic UAs (curl, python-requests) can get 403 or IP-blocked without notice. Browser JS must use Api-User-Agent. Source: https://foundation.wikimedia.org/wiki/Policy:Wikimedia_Foundation_User-Agent_Policy
• Rate limits / etiquette: no hard read limit, but requests should be serial, not parallel; use gzip, maxlag for non-interactive jobs, cache data that does not change often, prefer GET. https://www.mediawiki.org/wiki/API:Etiquette . High-volume commercial reuse is meant to go through Wikimedia Enterprise (paid, with free community access): https://www.mediawiki.org/wiki/Wikimedia_APIs
• Languages: ~300 Wikipedia language editions; each is a separate host.
• License: article text CC BY-SA 4.0 (and GFDL). Images are per file (see Commons).
• Commercial use: allowed. Terms of Use section 7: "these licenses do allow commercial uses", if compliant with the license. https://foundation.wikimedia.org/wiki/Policy:Terms_of_Use
• Attribution: hyperlink or URL to the page (or a list of authors), a license notice, and a note if modified. For GAKN: show "Wikipedia" + article title linked to the article URL + "CC BY-SA 4.0" link. Snippets are excerpts; do not edit them silently.
• Caching: allowed (license-based); refresh periodically since articles change.
• Charging: GAKN may charge for its service; it must not restrict reuse of the CC BY-SA text itself (no DRM-style "this snippet is ours"). Keep attribution on every hit.
• Verdict: CONNECT NOW.

2. Wikidata

• Endpoints: https://www.wikidata.org/w/api.php?action=wbsearchentities and SPARQL https://query.wikidata.org/sparql
• Tested
  • https://www.wikidata.org/w/api.php?action=wbsearchentities&search=photosynthesis&language=en&uselang=en&limit=5&format=json -> 200, 5 entities (Q11982 first)
  • SPARQL SELECT ?item ?itemLabel WHERE { ?item wdt:P31 wd:Q5 . ?item wdt:P106 wd:Q169470 . SERVICE wikibase:label { bd:serviceParam wikibase:language "en" } } LIMIT 5 with format=json -> 200, 5 bindings
• Auth: none; Wikimedia UA policy applies.
• Limits (SPARQL): 60 s query timeout; 60 s of processing per 60 s per client (UA + IP); 30 error queries/min; 5 parallel queries per IP; 429 with Retry-After. https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual
• Languages: labels/descriptions in hundreds of languages (language= / uselang=).
• License: structured data (main, Property, Lexeme, EntitySchema namespaces) CC0; other page text CC BY-SA 4.0. https://www.wikidata.org/wiki/Wikidata:Licensing
• Commercial / caching / charging: CC0, so unrestricted; store indefinitely. Courtesy attribution "Wikidata" + link recommended.
• Use in GAKN: entity search and enrichment (multilingual labels, identifiers), not long-form content.
• Verdict: CONNECT NOW.

3. Wikimedia Commons

• Endpoint: https://commons.wikimedia.org/w/api.php
• Tested: https://commons.wikimedia.org/w/api.php?action=query&generator=search&gsrsearch=photosynthesis&gsrnamespace=6&gsrlimit=5&prop=imageinfo&iiprop=url|extmetadata&iiextmetadatafilter=LicenseShortName|Artist|ImageDescription|DateTimeOriginal&format=json&formatversion=2 -> 200, 5 files. Licenses observed: "Public domain", "CC BY-SA 3.0".
• Auth / limits: as Wikipedia.
• License: per file, returned in extmetadata.LicenseShortName (also LicenseUrl, Artist, AttributionRequired if requested). Structured data on Commons is CC0.
• Commercial use: depends on file; most are PD / CC BY / CC BY-SA (commercial OK). Filter out anything else.
• Attribution: author (Artist), license name + link, link to file description page. Required for CC BY*.
• Caching: metadata yes; hotlink thumbnails from upload.wikimedia.org rather than re-hosting media.
• Verdict: CONNECT NOW.

4. YouTube Data API v3

• Endpoint: https://www.googleapis.com/youtube/v3/search
• Tested: https://www.googleapis.com/youtube/v3/search?part=snippet&q=photosynthesis&type=video&maxResults=5 -> 403 PERMISSION_DENIED: "Method doesn't allow unregistered callers ... Please use API Key".
• Auth: Google Cloud API key (or OAuth). Requires the owner to create a Google Cloud project.
• Quota: search.list now has its own bucket: "Quota impact: 100 calls per day. A call to this method has a quota cost of 1 unit in the Search Queries quota bucket" (https://developers.google.com/youtube/v3/docs/search/list). Other endpoints share 10,000 units/day; videos.list costs 1 unit. Resets midnight Pacific. https://developers.google.com/youtube/v3/determine_quota_cost . 100 searches/day is far below what a search product needs; extensions require a compliance audit.
• Developer Policies (https://developers.google.com/youtube/terms/developer-policies), precise points:
  • Caching: Non-Authorized Data (what a search integration gets) may be stored "temporarily ... for as long as is necessary ... but not longer than 30 calendar days"; after that "delete or refresh". Statistics (e.g. subscriber counts) must not be stored more than 30 days. Must display "the most updated API Data available" and keep stored data consistent with current data (III.E.4).
  • Display / branding: any page showing YouTube content (search results, thumbnails, players) "must make clear to the viewer that YouTube is the source" using YouTube Brand Features per the Branding Guidelines; non-YouTube content must not look like it originates from YouTube (III.F.2).
  • Combining with other data: search clients "must not modify or replace the text, images, information, or other content of, the search results" and "must not merge or intermix results from sources other than YouTube and present them as YouTube search results" (III.C). Must not create new or derived data/metrics from API Data, or replace API Data with independently calculated data (III.E.4.h). A mixed GAKN result list is only acceptable if YouTube hits are unmodified and clearly branded as YouTube, and non-YouTube hits are clearly not.
  • Charging / commercial: Prohibited: "offer an API service that allows users to access any data or functionality directly or indirectly provided by the YouTube API Services" (I); "sell, purchase, lease, lend, convey, redistribute, or sublicense all or any portion of YouTube API Services"; "sell YouTube API Services or access to any components" without written approval; must not "charge users to watch content in an embedded YouTube player" (III.F/G). Ads on pages with YouTube data only if other independently valuable content is present. Permitted: selling the API Client itself and ad-supported clients (III.G.2).
  • Other obligations: users must be bound to YouTube ToS (link https://www.youtube.com/t/terms) and GAKN's privacy policy must mention YouTube API Services and link the Google Privacy Policy. No scraping.
• Why not: GAKN's core is an agent-facing search API. Returning YouTube data through it is "an API service that allows users to access data ... provided by the YouTube API Services", which is prohibited; paid tiers would also be "selling access". Plus 100 searches/day and the 30-day cache cap.
• Verdict: DO NOT CONNECT. (Possible later: human-only UI widget, separately branded, with owner's key and Google approval. That is an owner/product decision.) Use PeerTube/SepiaSearch for open video instead.

5. SepiaSearch (global PeerTube index, by Framasoft)

• Endpoint: https://sepiasearch.org/api/v1/search/videos (PeerTube search API shape; also /search/video-channels, /search/video-playlists)
• Tested
  • https://sepiasearch.org/api/v1/search/videos?search=photosynthesis&count=5 -> 200, total 72
  • https://sepiasearch.org/api/v1/search/videos?search=linux&count=5&languageOneOf=fr&nsfw=false&sort=-match -> 200, total 2,122
• Auth: none. Limits: no published numbers; PeerTube default is 50 calls / 10 s on /api/v1/* (https://docs.joinpeertube.org/api-rest-reference.html). Keep it low and cache.
• Terms: no API terms page found (sepiasearch.org/terms renders client-side; nothing extractable). Framasoft removes instances that violate French law from the index (https://framablog.org/2020/09/22/sepia-search-our-search-engine-to-promote-peertube/).
• Languages: per-video language.id; filter with languageOneOf.
• License: per video in licence.label (e.g. "Attribution - Share Alike", "CC BY-NC-ND 4.0", "Unknown"). Title/description belong to uploaders.
• Risk noted: some instances host re-uploads of third-party content (test hit: a TED talk on diler.tube). Show metadata + link only, always pass nsfw=false, keep a host blocklist.
• Commercial / caching / charging: metadata + link display is standard search-engine use; cache short-term (e.g. 24 h). Do not sell the videos; charging for GAKN search is fine.
• Verdict: CONNECT NOW.

6. PeerTube per-instance API

• Endpoint: https://{instance}/api/v1/search/videos. Tested https://framatube.org/api/v1/search/videos?search=linux&count=5 -> 200, total 1,254.
• Same fields as SepiaSearch; instance-local search (searchTarget=local) or federated. Each instance has its own ToS. Default limit 50 req / 10 s.
• Verdict: CONNECT NOW but route through SepiaSearch by default; direct instance calls only for whitelisted instances.

7. OpenAlex

• Endpoint: https://api.openalex.org/works (also /authors, /sources, /institutions, /topics)
• Tested: https://api.openalex.org/works?search=photosynthesis&per-page=5&mailto=aabscnonprofit@gmail.com -> 200, count 664,957. Also search=fotosíntesis&filter=language:es -> 200, 10,337.
• Auth / budget (changed in 2025-26): keyless requests work but the response carries meta.notice: "No API key, so this request shares one free daily budget ($0.10) with everyone on your network's IP address. The mailto parameter doesn't raise this limit." Each search cost meta.cost_usd = 0.001, so keyless = ~100 searches/day per IP, shared with everyone on that IP (fatal on Vercel/serverless egress). A free key gives "$1 of usage per day" (~1,000 searches/day), resets midnight UTC. Per-second: 10 (no key), 30 (free key / prepaid), 50 (annual). The old "polite pool" via mailto no longer raises limits. https://help.openalex.org/how-to-use-the-api/rate-limits-and-authentication , https://help.openalex.org/access/pricing/
• Owner action: create a free key at https://openalex.org/settings/api (or POST https://api.openalex.org/api-keys with the owner's email; the owner must click the email link). Pass api_key=....
• Languages: global, language field (ISO 639-1), filterable.
• License: CC0 ("public-domain license with no 'personal use only' carve-out"). Commercial OK, store indefinitely, attribution courtesy only.
• Note: abstract_inverted_index is present only for some works; abstracts originate with publishers, so display short snippets.
• Verdict: CONNECT WITH KEY (free key; adapter can be built and tested keyless today).

8. Crossref REST API

• Endpoint: https://api.crossref.org/works
• Tested: https://api.crossref.org/works?query=photosynthesis&rows=5&mailto=aabscnonprofit@gmail.com -> 200, total-results 44,266. Response headers: x-api-pool: polite-array, x-rate-limit-limit: 3, x-rate-limit-interval: 1s, x-concurrency-limit: 3.
• Auth: none. Add mailto= (polite pool). Back off on 429; cache; avoid repeated identical queries. Paid "Metadata Plus" for snapshots/SLA. https://www.crossref.org/documentation/retrieve-metadata/rest-api/ , https://www.crossref.org/documentation/retrieve-metadata/rest-api/tips-for-using-the-crossref-rest-api/
• Languages: multilingual records; language field when deposited.
• License: "almost none of the metadata is subject to copyright, and you may use it for any purpose", but "Some abstracts contained in the metadata may be subject to copyright by publishers or authors."
• Commercial / caching: yes, store indefinitely. Show abstracts only as short snippets (or omit), link to DOI.
• Verdict: CONNECT NOW.

9. arXiv API

• Endpoint: https://export.arxiv.org/api/query (Atom XML, not JSON)
• Tested: https://export.arxiv.org/api/query?search_query=all:photosynthesis&start=0&max_results=5 -> 200, opensearch:totalResults 437.
• Limits: "no more than one request every three seconds", single connection, across all your machines combined. https://info.arxiv.org/help/api/tou.html
• License: descriptive metadata (titles, abstracts, authors, ids, categories) CC0. E-prints are copyrighted by authors; do not store and serve PDFs; link to the abstract page.
• Branding: must not present GAKN as endorsed by arXiv.
• Languages: mostly English.
• Commercial / caching / charging: metadata unrestricted (CC0). Needs a server-side queue + cache because of the 3 s rule.
• Verdict: CONNECT NOW.

10. Europe PMC

• Endpoint: https://www.ebi.ac.uk/europepmc/webservices/rest/search
• Tested: https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=photosynthesis&format=json&pageSize=5&resultType=lite -> 200, hitCount 177,562. resultType=core adds abstractText, license (e.g. "cc by-nc-nd"), language, fullTextUrlList.
• Auth: none. Limits: no numbers; EMBL-EBI blocks usage that "prevents, or is likely to prevent, EMBL-EBI providing services to others". https://www.ebi.ac.uk/about/terms-of-use/ (europepmc.org doc pages returned 403 to our fetcher.)
• License / commercial: EMBL-EBI "places no additional restrictions on the use or redistribution of the data" beyond data owners' rights; "EMBL-EBI expects attribution (e.g. in publications, services or products)". Abstracts and full texts carry publisher/author rights; per-article license.
• Caching: metadata yes; abstracts as snippets.
• Languages: mostly English, language field.
• Verdict: CONNECT NOW (attribute "Europe PMC").

11. Semantic Scholar Graph API

• Endpoint: https://api.semanticscholar.org/graph/v1/paper/search
• Tested: https://api.semanticscholar.org/graph/v1/paper/search?query=photosynthesis&limit=5&fields=title,url,abstract,year,externalIds,openAccessPdf,publicationDate -> 429 on all three attempts ("Too Many Requests ... apply for a key"). The keyless pool is shared and saturated.
• License: https://www.semanticscholar.org/product/api/license . Limited, non-sublicensable license; must not "repackage, sell, rent, lease, lend, distribute, or sublicense the API"; attribution to "Semantic Scholar" required; S2 datasets carry their own licenses, some CC BY-NC (non-commercial). Commercial use not addressed explicitly.
• Assessment: redundant with OpenAlex (CC0) for GAKN's purposes; licensing ambiguity for an agent-facing commercial API.
• Verdict: CONNECT WITH KEY (owner applies for a key at https://www.semanticscholar.org/product/api#api-key-form and confirms commercial terms). Low priority.

12. Internet Archive (advancedsearch)

• Endpoint: https://archive.org/advancedsearch.php
• Tested: https://archive.org/advancedsearch.php?q=photosynthesis&fl[]=identifier&fl[]=title&fl[]=description&fl[]=date&fl[]=language&fl[]=licenseurl&fl[]=mediatype&rows=5&page=1&output=json -> 200, numFound 4,441.
• Auth: none. Automated clients must send a descriptive UA (tool, version, and for AI agents the model). Honor 429 / Retry-After, cache. https://archive.org/developers/bots.html
• License: "The Internet Archive does not assert any new copyright or other proprietary rights over any of the material in its database" (https://archive.org/developers/index.html). Items are a mix of public domain, CC, and in-copyright (lending library).
• Concern: the IA Terms of Use (https://archive.org/about/terms.php, JS-rendered, could not be extracted directly) have historically said access "is provided at no cost to you and is granted for scholarship and research purposes only". A paid commercial search layer conflicts with that wording; current text unverified.
• Verdict: DO NOT CONNECT until the owner reads the current ToU and decides (or asks IA). Technically trivial to add afterwards.

13. Open Library

• Endpoint: https://openlibrary.org/search.json
• Tested: https://openlibrary.org/search.json?q=photosynthesis&limit=5&fields=key,title,author_name,first_publish_year,language,ia,has_fulltext -> 200, numFound 1,747. Non-Latin queries work (q=Толстой -> 5,191).
• Limits: 1 req/s unidentified, 3 req/s with UA + email. https://openlibrary.org/developers/api
• Usage guidelines (verbatim): "they are not intended to serve as a data backend for third-party services"; "These APIs are not intended to serve as a bulk data backend or high-traffic commercial infrastructure"; "Please Do Not: ... Use Open Library as a backend for high-traffic commercial services". Prioritized: open-source/mission-aligned projects, library partners, research/education tools, book lookup tools for humans.
• Verdict: DO NOT CONNECT as a live backend. Owner options: email openlibrary@archive.org describing GAKN, or use the monthly data dumps.

14. Project Gutenberg via Gutendex

• Endpoint: https://gutendex.com/books/ (third-party JSON wrapper over the PG catalog)
• Tested: https://gutendex.com/books/?search=dickens -> 200, count 231 (32 per page). ?search=hugo&languages=fr -> 200, 43. One request timed out at 30 s; the public instance is slow.
• Auth: none. Gutendex itself says: "For long-term use, please run your own server" (open source).
• License: PG catalog data is public domain (https://www.gutenberg.org/policy/robot_access.html). Books flagged copyright: false are PD in the US. gutenberg.org website is "intended for human users only"; use catalog feeds/mirrors for bulk.
• Attribution: link to https://www.gutenberg.org/ebooks/{id}; do not use the "Project Gutenberg" trademark to brand modified texts.
• Verdict: CONNECT NOW (low volume, cached), then self-host Gutendex from the PG RDF catalog.

15. Europeana

• Endpoint: https://api.europeana.eu/record/v2/search.json
• Tested: without key -> 401. With the publicly documented demo key wskey=api2demo: https://api.europeana.eu/record/v2/search.json?wskey=api2demo&query=photosynthesis&rows=5 -> 200, totalResults 409. The demo key is for trying the API, not production.
• Auth: free personal API key (registration on pro.europeana.eu; owner action). Docs pages returned 403 / JS-only to our fetcher.
• License: Europeana metadata is CC0; each item's media rights are in rights (rightsstatements.org or CC URIs).
• Languages: all EU languages; language per item.
• Verdict: CONNECT WITH KEY (owner registers a free key).

16. Zenodo

• Endpoint: https://zenodo.org/api/records
• Tested: https://zenodo.org/api/records?q=photosynthesis&size=5 -> 200, hits.total 3,476.
• Limits: guest 60 req/min and 2,000 req/hour; search 30 req/min; anonymous page size max 25. https://developers.zenodo.org/
• License: "Unless specified otherwise, Zenodo metadata may be freely reused under the CC0 waiver" (https://about.zenodo.org/terms/). Files under depositor-chosen license (metadata.license.id, often absent).
• Verdict: CONNECT NOW.

17. DOAJ

• Endpoint: https://doaj.org/api/search/articles/{query} (redirects/pages to /api/v4/; use https://doaj.org/api/v4/search/articles/{query} directly). Also /search/journals/.
• Tested: https://doaj.org/api/search/articles/photosynthesis?pageSize=5 -> 200, total 20,739.
• Limits: "a rate limit of two requests per second on all API routes", bursts up to 5 queued. https://doaj.org/api/docs
• License: journal and article metadata CC0 (https://doaj.org/terms/). The "DOAJ" name/logo need permission; do not imply endorsement.
• Languages: many; bibjson.journal.language (e.g. ["CS","EN"]).
• Verdict: CONNECT NOW.

18. data.gov catalog

• Status change: the old CKAN endpoint https://catalog.data.gov/api/3/action/package_search now returns 404. The GSA proxy https://api.gsa.gov/technology/datagov/v3/action/package_search needs an api.data.gov key (DEMO_KEY returned 503 "no healthy upstream").
• New endpoint: https://catalog.data.gov/search (JSON; OpenAPI at https://catalog.data.gov/openapi.json, "Datagov Catalog" v0.1.0). Params: q, sort, per_page, after (cursor), keyword, org_slug, org_type, publisher, theme, has_download, spatial_*. Single dataset: /api/dataset/{slug_or_id}; human page https://catalog.data.gov/dataset/{slug} (200).
• Tested: https://catalog.data.gov/search?q=climate&per_page=5 -> 200, 5 results (no total count; cursor after).
• Auth: none. No published rate limits.
• License: federal metadata is US public domain; datasets federated from states/cities carry their own (often unspecified) licenses.
• Languages: English.
• Verdict: CONNECT NOW, with the adapter isolated because the API is v0.1.0 and recently replaced.

19. Hacker News (Algolia)

• Endpoint: https://hn.algolia.com/api/v1/search (and search_by_date)
• Tested: https://hn.algolia.com/api/v1/search?query=photosynthesis&hitsPerPage=5 -> 200, nbHits 2,925.
• Auth: none. Limits: docs page is JS-rendered; commonly cited as 10,000 requests/hour per IP (unverified).
• License: no published API terms; HN comments/stories are owned by their authors (Y Combinator HN terms). Display title + link + points/date; avoid storing comment bodies as GAKN content.
• Verdict: CONNECT NOW (titles and links only).

20. Stack Exchange API

• Endpoint: https://api.stackexchange.com/2.3/search/advanced
• Tested: https://api.stackexchange.com/2.3/search/advanced?order=desc&sort=relevance&q=photosynthesis&site=biology&pagesize=5 -> 200, 5 items, quota_remaining 299 of quota_max 300. Spanish site (site=es.stackoverflow) also 200.
• Limits: keyless 300 requests/day per IP; with a free app key 10,000/day; >30 req/s from one IP gets dropped/banned for 30 s to minutes; honor backoff. https://api.stackexchange.com/docs/throttle
• Terms (https://stackoverflow.com/legal/api-terms-of-use): "All Applications must ensure they visually indicate that the Stack Exchange Network is the source of the content"; apps indexable by search engines must follow the ToS attribution rules (link to question, author name + profile link, CC BY-SA). Content is CC BY-SA (content_license per item). The public ToS prohibits non-personal copying of public content "other than ... content made available via the Stack Overflow API" (https://stackoverflow.com/legal/terms-of-service/public). Stack sells "Stack Data Licensing" for AI uses; watch for policy changes.
• Verdict: CONNECT WITH KEY (owner registers a free app on stackapps.com; 300/day keyless is not enough).

21. Openverse (openly licensed images/audio)

• Endpoint: https://api.openverse.org/v1/images/ and /v1/audio/
• Tested: https://api.openverse.org/v1/images/?q=photosynthesis&page_size=5 -> 200, result_count 240. Each result includes a ready attribution string and license, license_url, foreign_landing_url, creator.
• Terms (https://docs.openverse.org/terms_of_service.html): must attribute CC works; apps must "prominently indicate that it was made using Openverse" and not imply endorsement; Openverse "reserves the right to charge fees for commercial uses" and heavy usage; no scraping; no multi-machine evasion of limits.
• Verdict: CONNECT WITH KEY (owner registers via /v1/auth_tokens/register/, verifies email, and states commercial use). Use license_type=commercial filter.

22. Library of Congress (loc.gov JSON API)

• Endpoint: https://www.loc.gov/search/?fo=json (and collection/format endpoints)
• Tested: https://www.loc.gov/search/?q=photosynthesis&fo=json&c=5 -> 200, 1,100 results.
• Limits: JSON API 20 requests/min, exceeding blocks the IP for 1 hour; may drop under load. https://www.loc.gov/apis/json-and-yaml/working-within-limits/
• License: no key; rights per item (rights field). LoC-created metadata is generally US public domain.
• Verdict: CONNECT NOW, behind a strict 1 req / 3 s limiter and cache.

23. PubMed E-utilities (NCBI)

• Endpoint: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi + esummary.fcgi (two calls: ids, then summaries)
• Tested: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=photosynthesis&retmode=json&retmax=5&tool=gakn&email=aabscnonprofit@gmail.com -> 200, count 71,511.
• Limits: 3 req/s without key, 10 req/s with free NCBI key; include tool and email. Abstracts may be copyrighted by publishers.
• Verdict: CONNECT NOW (optional; Europe PMC already covers PubMed in one call).

24. dblp

• Tested: https://dblp.org/search/publ/api?q=photosynthesis&format=json&h=5 -> 202 with an HTML "Making sure you're not a bot!" (Anubis proof-of-work) page. Data is CC0, but the live API is not reachable for server-side clients.
• Verdict: DO NOT CONNECT (live); dumps possible later. OpenAlex/Crossref cover CS literature.

---

Response shapes for CONNECT NOW adapters

Field paths for title / url / snippet / date / language / license.

Source	Results array	title	url	snippet	date	language	license
Wikipedia REST	pages[]	title	build https://{lang}.wikipedia.org/wiki/{key}	excerpt (HTML with <span class="searchmatch">), description	none in search; use summary timestamp	from host {lang}	constant CC BY-SA 4.0
Wikipedia Action	query.search[], total query.searchinfo.totalhits	title	build from title / pageid	snippet (HTML)	timestamp (last edit)	from host	CC BY-SA 4.0
Wikidata	search[]	label (or display.label.value)	"https:" + url / concepturi	description	none	display.label.language / match.language	CC0
Commons	query.pages[]	title	imageinfo[0].descriptionurl (media imageinfo[0].url)	imageinfo[0].extmetadata.ImageDescription.value (HTML)	extmetadata.DateTimeOriginal.value	n/a	extmetadata.LicenseShortName.value (+ Artist.value)
SepiaSearch / PeerTube	data[], total total	name	url (Sepia) or https://{instance}/w/{shortUUID}	description (truncated)	publishedAt	language.id	licence.label (licence.id may be null)
Crossref	message.items[], total message.total-results	title[0]	URL (doi.org)	abstract (JATS XML, often null)	issued.date-parts[0] (may contain null) or created.date-time	language	license[].URL (often absent)
arXiv (Atom XML)	feed/entry, total opensearch:totalResults	entry/title	entry/id or link[@rel=alternate]/@href	entry/summary	entry/published	n/a (mostly en)	metadata CC0; paper license not in API
Europe PMC	resultList.result[], total hitCount	title	build https://europepmc.org/article/{source}/{id} or doi	abstractText (resultType=core)	firstPublicationDate / pubYear	language (core)	license (core, e.g. "cc by")
Gutendex	results[], total count, paging next	title	build https://www.gutenberg.org/ebooks/{id}	summaries[0]	none (author years only)	languages[]	copyright (false = US PD)
Zenodo	hits.hits[], total hits.total	metadata.title	links.self_html (+ doi)	metadata.description (HTML)	metadata.publication_date	metadata.language (ISO 639-3, often absent)	metadata.license.id (often absent)
DOAJ	results[], total total	bibjson.title	bibjson.link[0].url (or DOI in bibjson.identifier[])	bibjson.abstract	bibjson.year (+ month)	bibjson.journal.language[]	article-level not in this record; journal is OA, metadata CC0
data.gov	results[], cursor after	title	build https://catalog.data.gov/dataset/{slug}; source dcat.landingPage	description	dcat.modified / dcat.issued	n/a (en)	dcat.license (often absent)
HN Algolia	hits[], total nbHits	title	url (or https://news.ycombinator.com/item?id={objectID})	story_text (often null)	created_at	n/a	none (author-owned)
LoC	results[], total pagination.of	title	url	description[0]	date	language[]	rights
PubMed (esummary fields per NCBI docs, not tested)	esearchresult.idlist[] then esummary result[uid]	title	https://pubmed.ncbi.nlm.nih.gov/{uid}/	none (efetch for abstracts)	pubdate	lang[]	none
OpenAlex (key)	results[], total meta.count	display_name	doi or primary_location.landing_page_url (or id)	rebuild from abstract_inverted_index (often null)	publication_date	language	primary_location.license, open_access.oa_status