ThingsBoard UI contract — data fidelity¶
UI Contract Coverage verifies that every endpoint the
real ThingsBoard UI calls reaches a real handler: 284 routed, 89 declared
no-goal, 0 gap. That check only ever asked "does this respond with the right
status and shape?" — never "is the data in the body real?" A handler can be
fully "routed" and still return hardcoded zeros, silently re-list instead of
create, or answer 200 to a DELETE that deleted nothing. This document is
that second, orthogonal axis: for each of the 284 routed entries, does it do
real work, or does it fake it?
Background and methodology: ADR-0002.
Result¶
| count | |
|---|---|
| Audited | 284 / 284 routed entries |
confirmed-gap |
28 |
confirmed-gap-conditional |
9 |
out-of-scope (deliberate, documented non-answer) |
~90 |
verified (real, complete work) |
~150 |
needs-live-check (undecidable statically) |
2 — one resolved as verified, one open |
| Fixed so far | 30 |
The 37 real findings cluster almost entirely in internal/system/*.go (8 of
9 files carry zero dedicated tests) and inline closures in api.go — exactly
where ADR-0002's Tier-0 signal
predicted risk would concentrate, not spread evenly across all 284.
Fixed: pass 1 — endpoints that faked success¶
Four endpoints answered 200/data to a request that did nothing — worse than
an honest stub, because the caller has evidence of success for an action that
never happened. All four now fail honestly (404/405), matching the
"answered honestly" convention UI Contract Coverage
already established for its 12 endpoints that return 501 rather than a
misleading 200.
| Endpoint | Was | Now | Why |
|---|---|---|---|
DELETE /api/calculatedField/{id} |
200 OK |
404 (matches sibling GET) |
No calculated_field row is ever created (POST doesn't persist either — see backlog); GET already honestly says "not found." DELETE agreeing is the fix, not new delete logic. |
DELETE /api/widgetType/{id} |
200 + the row's data |
405 Method Not Allowed |
Route registered method-agnostic (comment: "read-only") but nothing enforced that. No DELETE FROM widget_type exists anywhere in the repo. |
DELETE /api/widgetsBundle/{id} |
200 + the row's data |
405 Method Not Allowed |
Same bug, same fix, same "not supported yet" convention already used for POST/PUT /api/widgetsBundle. |
DELETE /api/ruleChain/{id} |
200 + the chain's data |
405 Method Not Allowed |
HandleRuleChainByID has no method branch; only ever SELECTs. No DELETE FROM rule_chain exists anywhere in the repo. |
Files: flow-core/internal/system/feature_handlers.go,
flow-core/internal/widget/widget.go, flow-core/api.go. Tests:
flow-core/internal/system/calculated_field_delete_test.go,
flow-core/internal/widget/widget_delete_test.go,
flow-core/rulechain_delete_test.go — each asserts the honest status, red
against the prior code, green now. None touch Postgres: all four fixes
reject the bad method before any DB access.
Fixed: pass 2 — real data that existed but was never read¶
Eight P2 findings where a real, populated subsystem already existed and the handler simply never called it. Each of these is a wire-up, not new behaviour — the data, the tables and the query helpers were all already there.
| Endpoint | Was | Now |
|---|---|---|
GET /api/usage |
transportMessages: 0, every max*: 0 |
transportMessages from the ts_kv snapshot internal/usage already writes every minute (the multi-replica-safe source, up to 60s stale — deliberately chosen over the process-local atomic); the five quota maximums from quotas.LimitsFor, TTL-cached. 0 still legitimately means "unlimited". |
GET /api/oauth2/client/infos |
[] always |
reads the real oauth2_client table, same rows GET /api/oauth2/client already served |
GET /api/oauth2/config/template |
[] always |
reads oauth2_client_registration_template, seeded at every boot by internal/bootstrap.loadOAuth2Templates |
GET /api/tenant/dashboard/home/info |
{nil, true} always, no DB read |
reads tenant.additional_info |
POST /api/tenant/dashboard/home/info |
no method branch — silently discarded the selection | persists into tenant.additional_info, merging so it can't clobber other keys |
GET /api/dashboard/home |
empty 200 always |
reads tb_user.additional_info.homeDashboardId and forwards to ByID; still empty when genuinely unconfigured |
GET /api/admin/featuresInfo |
oauthEnabled: false always |
reflects real OIDC configuration |
POST /api/assets, POST /api/entityViews |
silently re-listed instead of creating | dispatch to asset.Save / entityview.Save, the real create handlers already wired at the singular routes |
PUT /api/image/import |
response omitted 8 fields it had already computed and written | returns the full shape, matching sibling ImageUpload |
Tests: internal/system/{usage,oauth2_infos,tenant_dashboard_home,features_info}_test.go,
internal/dashboard/home_test.go, internal/resource/image_import_test.go,
assets_entityviews_post_test.go. Most seed a real Postgres via
FLOW_TEST_PG_DSN (skipping when unset, the repo's existing convention) —
they were not run against the local operational Postgres, since they
DROP TABLE. features_info_test.go and the pass-1 tests need no DB and run
unconditionally.
The /api/usage test covers the quota fields only; transportMessages
resolves through telemetry.DeviceKVLatest, which needs a live GreptimeDB
connection, so its fallback-to-0 path is what the unit test exercises. Verify
that field end-to-end on a live cluster.
Fixed: pass 3 — entity-query filters that silently returned nothing¶
POST /api/entitiesQuery/find dropped whole classes of query on the floor: an
unhandled entity type or filter name hit a default case that logged a
server-side WARN and answered an empty 200. The caller saw "no results",
not "unsupported" — the two are indistinguishable to a UI widget.
| What | Was | Now |
|---|---|---|
resolveEntity for CUSTOMER / USER / ENTITY_VIEW |
nil — any singleEntity/entityList filter naming one was dropped |
real tenant-scoped lookups against customer / tb_user / entity_view. USER names by full name, falling back to email, matching the user list's own precedence. |
entityName filter |
unhandled → empty | name-prefix search over a closed allow-list of entity types; an unsupported type is refused rather than guessed at |
entityViewType filter |
unhandled → empty | the entity_view equivalent of the existing deviceType/assetType filters |
deviceSearchQuery / assetSearchQuery / entityViewSearchQuery |
unhandled → empty | relation walk from the root via topology.NeighborsTenant — the same tenant-scoped primitive relationsQuery uses, so a foreign root yields nothing rather than leaking — then narrowed by the requested entity type and optional subtypes |
stateEntityOwner filter |
unhandled → empty | resolves the entity's owning customer, falling back to the tenant (including for TB's nil-UUID "no owner" sentinel, which is stored instead of NULL) |
Test: internal/entityquery/entityquery_legacy_filters_test.go. Unlike the
earlier passes, this one was run against a real Postgres — its harness
follows the existing entityquery_relations_test.go pattern of creating a
throwaway schema and dropping it with CASCADE, so it never touches real
tables. All 12 subtests pass, including the two cross-tenant isolation
assertions. Running it for real caught a genuine gap in the test schema
(NeighborsTenant's union needs the profile tables to exist), which a
skipped test would have hidden.
Fixed: pass 4 — alarm fields that disagreed with their own list¶
Three P3 findings on paths that otherwise worked. All in
internal/tenant/tenant_handler.go.
| What | Was | Now |
|---|---|---|
GET /api/alarm/{id} |
originator.entityType hardcoded "DEVICE"; status derivation collapsed the four states so a cleared-but-unacknowledged alarm read CLEARED_ACK; customerId/assigneeId/propagate*/ackTs/clearTs/assignTs/originator name absent |
reads the same columns and derives the same fields as the sibling list query, so opening an alarm and seeing it in a list can no longer disagree. Status derivation is now one shared alarmStatus helper. |
assignee on both reads |
unconditional nil even with a real assignee_id — the field api.go documents as v2's addition over v1 |
resolved from tb_user, tenant-scoped |
POST /api/alarmsQuery/find |
never read its body — answered the whole tenant's alarms however narrowly the caller scoped the request | honours the posted entity filter (all the shapes TB has shipped) and pageLink, scoping through the entity_alarm join handleAlarmsByDevice already uses. GET /api/alarms shares the handler and is unchanged; a malformed reference degrades to unscoped rather than erroring. |
Test: internal/tenant/alarm_fidelity_test.go, run against a real Postgres
via the throwaway-schema pattern. 9 subtests, including cross-tenant
isolation on the assignee lookup.
A bug this pass introduced and the test caught. The first version
resolved each assignee inside the rows.Next() loop. That holds a pooled
connection while asking for another: on a single-connection pool it
deadlocks outright, and on any pool it is an N+1 that keeps a connection
busy for the whole page. The test hung rather than failed, which is what
made it obvious. Assignee ids are now collected during the scan and
resolved in one pass after rows.Close(), so distinct assignees cost one
lookup each and repeats collapse. Worth noting because the deadlock would
not have surfaced in production (PG_MAX_OPEN_CONNS is 30) — it would have
shown up as pool pressure under load instead.
Fixed: pass 5 — widget authoring and notification persistence¶
The two largest remaining P2 items. Both had real tables all along; only the wiring was missing.
POST/PUT /api/widgetType (internal/widget/widget.go) never read the
body or checked the method, so a save fell through to the GET branches and
answered the same 400 about missing query params — custom widget authoring
from the UI was inert even though widget_type is a table the seeder already
writes to. It now persists, scoped to the caller's tenant: an update is
refused unless the row already belongs to them, which also protects the
shared system-tenant widget catalogue the seeder installs.
The notification entities (internal/system/notification_crud.go, new)
all routed through saveJSONEntity, which echoes the posted JSON back with a
generated id and never touches the database. Every list endpoint then
correctly reported empty — nothing had ever been written. Now real:
| Endpoint | Notes |
|---|---|
POST /api/notification/target + GET /api/notification/targets |
round-trips through notification_target |
POST /api/notification/template + GET .../templates |
round-trips through notification_template |
POST /api/notification/rule + GET .../rules |
round-trips through notification_rule. template_id is NOT NULL with an FK, so a missing or unknown template is refused with a 400 up front rather than surfacing as a constraint-violation 500, and a template belonging to another tenant is refused with 403. |
POST /api/notification/request + GET .../requests |
records the request. Status is SCHEDULED, not SENT — the old code stamped SENT without storing or sending anything. There is still no email/SMS transport here, so claiming delivery would be exactly the fabricated success this audit exists to remove. |
PUT /api/notifications/read |
performs the real UPDATE, scoped to the calling user so one recipient cannot mark another's. Nothing produces rows in notification yet, so today it legitimately affects zero rows — but unlike the unconditional 200 it replaces, it becomes correct on its own the moment a producer exists. |
Tests: internal/widget/widget_save_test.go,
internal/system/notification_crud_test.go, both run against a real Postgres
with throwaway schemas. They cover the round trips plus cross-tenant
isolation on every writer.
Still not fixed here, deliberately. POST /api/notification/request/preview
still reports totalRecipientsCount: 0: resolving a target's configuration to
an actual recipient list is new logic, not a rewire, and belongs with delivery.
DELETE on the target/template/rule routes still answers 200 — those routes
carry no {id}, so they cannot identify a row to delete; the delete path is
unrouted rather than implemented, and changing its status is a contract
decision the UI manifest gives no evidence for either way.
A pre-existing test-isolation issue this surfaced. Running several
packages with FLOW_TEST_PG_DSN set fails intermittently, because the older
test harnesses (internal/tenant/tenant_handler_authz_test.go,
internal/system/admin_settings_authz_test.go, and others) DROP TABLE on
shared public tables, and Go runs packages in parallel by default. go test
-p 1 passes cleanly. It never surfaces in CI, where the DSN is unset and
those tests skip. Newer tests (this pass and passes 3-4) use throwaway
schemas and do not participate in the conflict — converting the older ones
would be a worthwhile follow-up.
Fixed: pass 6 — public sharing now fails honestly¶
POST /api/customer/public/{asset,dashboard,device,entityView}/{id} and
DELETE /api/customer/public/dashboard/{id} answered 403 "Customer belongs
to another tenant." That message is untrue — public names no customer at
all — and it reads as a permissions problem an operator could fix, rather
than a capability that isn't here. They now answer 501 "Public sharing is
not enabled on this platform", matching what /api/auth/login/public
already says for the same reason.
Why not implement it. The feature is missing in two layers, not one:
- Assignment. TB gives each tenant a real
customerrow titledPublicwithis_public = true; sharing means assigning the entity to it. No code in this repo has ever created that row — five places only excludetitle != 'Public'from listings and counts, assuming it exists. AndcustomerBelongsToTenantwhitelisted TB's nil-UUID sentinel rather than the literal segmentpublic, so even the intent was mis-encoded. Even granting the check, the next statement isUPDATE device SET customer_id = 'public', which fails as an invalid UUID. - Viewing. There is no way to open what was shared.
/api/auth/login/publicis a documented 501,internal/auth/jwt.gohardcodesisPublic: falsein every token it mints, and no route serves a dashboard without one.
Fixing only (1) would mark entities "shared" that nobody can open — the fabricated capability this audit exists to remove. Implementing (2) is not "mint a token": see the authorization note below.
Test: internal/customer/assign_test.go
(TestAssignToPublic_IsNotImplemented). One of its cases guards a subtlety —
the UI contract pins 404 for these paths, because its probes name an
entity that does not exist, so the 501 must sit after the entity lookup or
the contract check breaks. That ordering is asserted directly rather than
left to a live contract run: the deployed test cluster runs an older image,
so ui-contract-check against it would not exercise this change at all.
Related finding: customer-scoped authorization does not exist¶
Surfaced while researching the above, and larger than the finding that
prompted it. No handler in flow-core reads claims["customerId"]. Every
read — dashboards (internal/dashboard/dashboard.go), telemetry
(internal/telemetry/reader.go), the WS plane — is scoped by tenant only,
with a SYS_ADMIN bypass. A CUSTOMER_USER token therefore sees everything in
its tenant, not just its customer's entities.
A comment in api.go claimed the opposite ("the JWT scope … determines what
dashboard.ListByTenant filters internally"); it has been corrected in
place, since a false comment about an authorization boundary is worse than no
comment.
This is not exploitable by an outsider — every path still requires a valid token for the tenant, and this platform does not currently issue customer-scoped tokens to anyone (the demo seeds one customer user; OIDC and password login both mint tenant-scoped tokens). It is recorded here because it is the real reason public sharing cannot simply be switched on: an anonymous public token would need a boundary that has not been built, and building it is security-critical work on a deny-by-default surface that was deliberately hardened. It deserves its own review before any customer-scoped or public token is issued.
Fixed: pass 7 — the last mechanical findings¶
GET /api/noauth/userPasswordPolicy returned fixed TB defaults
unconditionally, so a policy a SYS_ADMIN had saved was never shown. It now
reads passwordPolicy out of the real securitySettings admin_settings row,
overlaying rather than replacing so a stored policy that omits a field keeps
the default for it. Only that sub-object is read: this endpoint is public
(/api/noauth/), while the rest of securitySettings — lockout thresholds,
the lockout notification email — is SYS_ADMIN-scoped in HandleAdminSettings
and must not leak through it. A test asserts that non-leak explicitly.
Caveat worth knowing: nothing in flow-core validates a password against this policy — it is advisory, enforced client-side. Reading the real values changes what the UI displays and checks, not what the server accepts. Server-side enforcement is a separate change with real lockout risk for existing accounts.
POST /api/notification/request/preview reported
totalRecipientsCount: 0 unconditionally. Targets are real rows since pass 5,
so the count is now computed per target. The usersFilter vocabulary and
field names were extracted from the deployed UI bundle
(thingsboard/tb-web-ui:4.3.1.1 →
configuration.usersFilter.{type,usersIds,customerId,filterByTenants,tenantsIds,tenantProfilesIds}),
the same source docs/UI_CONTRACT_COVERAGE.md's endpoint catalogue came
from — not guessed. ALL_USERS, TENANT_ADMINISTRATORS, CUSTOMER_USERS
and USER_LIST resolve; a filter that needs a subsystem this platform lacks
(system administrators, cross-tenant fan-out) is omitted from the response
rather than reported as 0, because a wrong count reads as authoritative
while an absent one reads as "not computed".
needs-live-check, resolved¶
POST /api/device/bulk_import — verified, not a gap. Its counts do match
what lands, across all three branches: fresh rows counted as created; a
repeat without mapping.update reported as per-row errors with nothing
duplicated; a repeat with mapping.update counted as updated and the row
actually changed (asserted, not assumed). Writing this test first produced a
failure, which turned out to be my own wrong assumption — I had expected
upsert semantics, while the implementation deliberately treats an existing
name as an error unless mapping.update is set. The behaviour was right and
the test was wrong; recorded because "the test failed" is not the same as
"the code is broken."
Priority backlog (confirmed, still open)¶
P1 — resolved as "answered honestly" in pass 6¶
Public sharing is not implemented and now says so (501) instead of failing as a misleading 403. Implementing it for real requires customer-scoped authorization that does not exist — see pass 6 above.
P2 — real backing data, never wired¶
All closed across passes 2, 3 and 5. The one remnant is
POST /api/notification/request/preview's totalRecipientsCount, which needs
recipient-resolution logic rather than a rewire — see pass 5.
P3 — all closed (passes 4 and 7)¶
needs-live-check — one open¶
GET/POST /api/queues(internal/system/stubs_handlers.go) — GET is a real query; POST is silently list-only. Unlike the assets/entityViews case this was not fixed, because it is a product decision rather than a missed wire-up: queue and consumer configuration is Helm/k8s-owned on this platform (rootCLAUDE.md), so DB-backed queue creation may be deliberately unsupported — in which case the honest answer is a declared no-goal, not an implementation. Needs a decision.
Confirmed out-of-scope (sample — not exhaustive)¶
The remainder of the 284 (roughly 90 entries) are deliberate, honest
non-answers, consistent with the same philosophy documented in
UI Contract Coverage's "answered honestly (12)"
section: 2FA, mobile QR, version-control/repository settings, Trendz
analytics, edge administration, mail transport, SMS transport, and dashboard
visit-tracking are all explicitly documented in code comments and/or
docs/API_REFERENCE.md as features this platform does not implement.
GET /api/components (rule-node catalogue) and the connections field of
rule-chain metadata are honest empties because flow-core ships no rule
engine (alarm detection runs in Bento/NATS instead, per api.go's own
comment). These were read and classified, not skipped — see the individual
batch transcripts referenced in ADR-0002 for the full per-entry list if
auditing this document's completeness.
Re-running¶
Data fidelity isn't (yet) part of the automated ui-contract-check — that
tool only asserts status/shape and is safe to run against a shared
pilot/production cluster for exactly that reason (fixed placeholder UUID, no
seeding). The P1–P3 findings above were confirmed by direct source reading
against the running code at commit time; re-verifying after future changes
means re-reading the cited handlers, or (preferred, going forward) adding a
seeded httptest case per finding the way this pass's four fixes did.