Ask Stroom AI: from a chat panel to a function you can call

Ask Stroom AI arrived in 7.11 as a preview feature. You open it from any table's menu and ask questions such as "Summarise this table", and it answers from the data in front of you. Three things about it have held from the start:

• Any OpenAI-compatible model works. You are not tied to one provider, and a model you host yourself is a first-class option. That matters when the data cannot leave your network.
• Models are configured as documents. An OpenAIModel document holds the base URL, the model id, the name of the API key to use, the context window, and HTTP client settings.
• It is a preview feature. Expect it to change between releases.

7.13: a proper conversation

The 7.13 betas turned the dialog into something you can work in:

• The chat can be docked as a panel.
• History is kept, and messages and attachments can be reopened or deleted.
• Tables can be attached to a conversation by name, so the model can refer to them by source.
• Large tables are analysed in batches and the results merged, rather than the request simply being too big.
• Use of Ask Stroom AI is recorded in the event log.
• Ask Stroom AI queries can be cancelled from the Task Manager (#5758).

Chat history is stored in the database, which is why a new ai configuration branch appeared in 7.13.

7.14-beta.1: AI inside the pipeline and the query

Two additions take the model out of the chat window (#5751, #5761):

• ask-ai() in XSLT and askAi() in StroomQL send a question to a named model and return its answer. The StroomQL form is askAi(model, message[, systemPrompt]).
• AI summaries in reports. A scheduled report can now carry a written summary of its own results.

One caution: the model is asked once per value. Called over a million rows, askAi() makes a million requests. Use it on aggregated or filtered results, not raw event streams.

Where this is going

Everything here is still a preview, and the direction is clear: the model becomes another function Stroom can call, pointed at whichever model you choose. If you want help choosing and hosting one inside a closed network, talk to us.