AI product backends
Prompts are private data. Store them like it.
Conversation history, retrieval, async pipelines, and live job status — as backend features under one access model rather than four systems you operate.
- Who this is for
- Teams shipping products with an inference layer
- What you already have
- Model access and a product idea
- What you need
- Storage, retrieval, pipelines, and cost control
The problem
An AI product has backend requirements a CRUD tutorial never covers, and each one is a separate infrastructure project by default: a queue for asynchronous inference, a vector store for retrieval, a socket layer so users are not staring at a frozen spinner, and rate limiting so one enthusiastic user does not spend your monthly inference budget in an afternoon. Meanwhile the most sensitive table in the product is the one holding every prompt your users have ever written.
What you would normally build
A queue and workers, a standalone vector database with its own copy of sensitive data and its own sync bugs, a WebSocket server, a rate limiter, and the glue between them — before shipping the model behaviour that actually differentiates you.
What Backenly does
Model
Conversations and runs, with isolation proven
Describe the shape — documents, runs, a status, an owner — and the isolation rule. The post-build check signs in as a second user and asserts they receive zero rows of the first user's data. On a table holding prompts, that is the check you most want to exist and least want to write.
Retrieve
Vectors beside the rows they describe
pgvector columns live in the same schema, and similarity queries run under the same user context as any other read. A retrieval that ignores row-level security is a breach with extra steps — user A's question surfacing user B's documents as context. Keeping vectors in the policy-enforced database is the structural fix.
Process
Database events are the queue
Functions attach to on_db_insert, on_db_update, on_db_delete, on_signup, a cron schedule, or an HTTP endpoint. "When a document is added, summarise it and write the summary back" is a function on an event, not a worker fleet.
Stream
Job status is a row
Make status a column and subscribe to changes on it over Server-Sent Events. Every writer — your inference layer, a trigger function, the dashboard — feeds the same stream, and there is no socket server to run.
Bound
Per-key rate limits and real request metrics
Rate limits on the runtime API, and latency percentiles and error rates computed from one request-log source of truth, so "which endpoint went hot" is a glance rather than a log dive.
const unsub = backend.realtime.subscribe('ai_runs', (event) => {
if (event.type !== 'update') return
showStatus(event.data.status) // queued → running → done
if (event.data.status === 'done') {
renderOutput(event.data.output)
unsub()
}
})What you end up with
One system, one access model, one bill: PostgreSQL with pgvector, REST with verified isolation, event and cron functions with metered invocations, and SSE for live status. Your differentiation stays in the model layer, which is the only part nobody else can do for you.
Who owns what
Backenly does
- Enforces per-user isolation in the database, including on similarity queries.
- Runs event, cron, and HTTP functions with metered invocations.
- Streams row changes over SSE through a shared listener hub.
- Applies per-key rate limits and records request-level metrics.
You own
- Model choice, prompts, evaluation, and inference cost.
- Chunking and embedding strategy — the platform stores and queries vectors, it does not design your retrieval.
- Deciding what the pipeline should actually do at each step.
- Handling provider failures and retries inside your function logic.
What this is built on
| Capability | What it does here |
|---|---|
| pgvector | Embedding columns in your workspace schema; similarity queries run under the same user context as any other read. |
| Functions | on_signup, on_db_insert / on_db_update / on_db_delete, cron, http, and manual triggers. |
| Invocation quota | Metered per plan and enforced at execution — 10,000 function runs a month on Free, 2 million on Pro. |
| Realtime SSE | PostgreSQL LISTEN/NOTIFY through a shared listener hub, with auto-reconnect. Row change events. |
| Rate limits | Per-key limits on the runtime API, with the ceiling set by plan. |
| Behavioural verification | Two-user isolation asserted against the live runtime after a build. |
Known limitations
- Realtime carries row change events, not token streams. Stream tokens from your own inference endpoint; use SSE for status transitions.
- Vector search is not on the advertised 20-tool surface. Ask for it through backend_chat rather than a named tool.
- Event triggers are a paid capability — the Free plan seeds zero triggers per project. Check the pricing page before designing a pipeline around them.
- Function invocations are metered and enforced. A hot pipeline hits the plan quota and is refused rather than silently billed.
- The platform does not manage your inference spend. Rate limits bound request volume; they do not know what a request costs you.
Common questions
Can I use my own model provider?
Yes. Backenly stores and serves data over REST, so your inference layer reads and writes like any other client. Functions can also call providers directly through a registry surface, with the credential verified against the provider at connect time rather than merely stored.
Do I need a separate vector database?
Not for product-stage retrieval. pgvector keeps embeddings next to the rows they describe, so similarity search composes with your existing filters and with row-level security. A standalone vector store is a second system to operate, a second copy of sensitive data to secure, and a class of sync bug you do not need yet.
How do I show progress on a slow generation?
Make the job a row with a status column, update it as the work advances, and subscribe to changes on that row over SSE. Because the events come from the database, any writer feeds the same stream.
Try it on one free project
No credit card. Connect your agent over MCP and judge it by the verification evidence.