Bigdata Skill

Bigdata.com SDK + REST Toolkit

Get the structured substrate the Bigdata.com MCP server doesn't hand over. The MCP returns clean prose and pre-synthesized tearsheets, but its search tool gives chunks with no per-chunk sentiment or entity spans, and its tearsheets give aggregate values — not the fiscal-period time series, universe screener, or per-field JSON you'd build a pipeline on. The official bigdata-client SDK plus a thin REST passthrough over the same backend, same JWT reach the official /v1/* endpoints that hold it. This skill bundles a toolkit that does exactly that — already debugged, already cost-guarded — so you don't re-pay the discovery cost.

The core problem this solves (read this first)

The Bigdata MCP server answers "what's the sentiment around NVIDIA?" with a readable paragraph or a pre-synthesized tearsheet — genuinely useful for a chat turn. But the moment you need the machine-readable substrate to build a pipeline on, the MCP doesn't hand it over:

• its search tool returns chunks with text + relevance only — no per-chunk sentiment number, no entity character spans;
• its tearsheets give aggregate values (a single sentiment score, a summary of estimates) — not a fiscal-period time series you can compute on, a universe screener, or per-field JSON.

The fix is a general pattern, not a Bigdata trick:

When an MCP data source returns only synthesized output but you need the structured fields underneath, drop to the vendor SDK or REST. MCP optimizes for a chat turn, not a pipeline.

Crucially, for Bigdata these structured fields are official, publicly documented REST endpoints (docs.bigdata.com/api-reference/...), not a hidden backend — and Bigdata is sunsetting the SDK (EOL 2026-12-31) in favour of this REST API, so the REST layer here is the forward-compatible path, not a hack. The SDK (bigdata_client.Bigdata) covers search + knowledge-graph; bd._api.http reaches every /v1/* endpoint the SDK never wrapped. The bundled bigdata_toolkit packages both behind one BigdataClient.

When to use this skill

Trigger on any of these, in any language:

• The user is using Bigdata.com / RavenPack and the MCP result feels thin — "where's the sentiment score?", "I need entity-level data", "the calendar".
• They want forward / structured financials for a ticker: analyst estimates, earnings or event calendar, earnings surprise, analyst ratings, price targets, a company screener / universe.
• They want annotated news chunks with numeric sentiment + entity spans, or a sentiment time series / co-mention graph.
• They mention a bd_v2_ API key, rp_entity_id, query_unit / chunk cost, bigdata-client, or "the bigdata MCP isn't enough".
• They're building an investment-research dataset and need a reusable, cost-aware data-pull layer rather than one-off MCP calls.

Setup (one time)

1 — API key (never hardcode it). The client fail-fasts if it's missing:

export BIGDATA_API_KEY=bd_v2_xxxxxxxx

2 — An isolated Python env with the official SDK. The bundled toolkit imports bigdata_client; install it once:

uv venv .venv --python 3.12
uv pip install --python .venv/bin/python bigdata-client
# Behind a slow/blocked PyPI (e.g. mainland China) add a mirror, and unset any
# outbound proxy for the install step so uv reaches the index directly:
#   --index-url https://pypi.tuna.tsinghua.edu.cn/simple

3 — Outbound proxy (only if your network needs one to reach api.bigdata.com). Two equivalent options — the official SDK accepts both: an env var, or BigdataClient(proxy=...) in code. The env var is simplest:

export HTTPS_PROXY=http://<host>:<port>     # plus WSS_PROXY for chat/WebSocket

If a proxy does TLS interception (self-signed CA) and you hit SSL handshake errors, the official fix is BigdataClient(verify_ssl="<proxy-CA>.pem") — not blind retries.

4 — Make the bundled package importable by putting this skill's scripts/ on PYTHONPATH (or sys.path.insert(0, "<this-skill>/scripts")).

Smoke-test the whole path (entity resolve + quota are free; --with-search adds one ~1 query_unit chunk search):

BIGDATA_API_KEY=bd_v2_xxx PYTHONPATH=scripts .venv/bin/python scripts/probe_example.py

Quickstart

import sys
sys.path.insert(0, "<this-skill>/scripts")          # so `import bigdata_toolkit` resolves
from bigdata_toolkit import (
    BigdataClient, EntityResolver, AnnotatedSearcher,
    StructuredDataREST, CostTracker, CostModel, rc,   # rc = SSL-retry wrapper
)

c  = BigdataClient()                                  # SDK + REST escape hatch, one object
er = EntityResolver(c)
nvda = rc(lambda: er.resolve_id("NVIDIA", country="US"))   # -> 'E09E2B'  (rp_entity_id is the gateway key)

# --- Structured financials the MCP does NOT expose (REST escape hatch) ---
rest = StructuredDataREST(c)
est  = rc(lambda: rest.analyst_estimates(nvda, period="quarter", limit=5))  # forward consensus
surp = rc(lambda: rest.latest_surprise(nvda))                               # last EPS/revenue surprise
cal  = rc(lambda: rest.events_calendar(nvda, categories=["earnings-call"],
                                       start_date="2026-06-01", end_date="2026-12-31"))

# --- Annotated chunks the MCP STRIPS: sentiment + entity spans (cost-guarded) ---
s    = AnnotatedSearcher(c)
docs = rc(lambda: s.search_entity(nvda, keyword="data center", chunk_limit=10))
# each chunk dict: {"sentiment": float, "entities": [{"key": rp_id, "start", "end"}], "text", ...}

# --- Always know your spend (chunk-billed; see Cost discipline) ---
ct = CostTracker(c); ct.snapshot()
# ... run a batch ...
print(ct.delta())     # {'delta_chunks':..., 'delta_query_units':..., 'usd_fast':...}

Wrap every network call in rc(lambda: ...) — a first-handshake SSL: UNEXPECTED_EOF is common and the SDK's internal retry doesn't cover it.

Routing — which capability answers the question

income/balance/cash-flow/daily-prices/dividends/revenue-segments return {fields, values} — wrap them in fields_values_to_records() to get [{field: value}]. The *_ttm / company_profile endpoints are already flat. All structured endpoints above are free (0 chunks) except connected_entities and AnnotatedSearcher (chunk-billed).