Figure 1 depicts the MPAI-AIF Basic Profile Reference Model. You can see the Controller – the brain of the system – and the MPAI-AIF APIs enabling the Controller to obtain AIWs/AIMs from the MPAI Store, the place where implementers can submit their implementations for distribution after they have been tested for conformance with the standard and verified for security. Once the AI Framework is equipped with the desired domain-specific processing capabilities, the User Agent can activate the Controller, and the AIMs can call it via the appropriate APIs.

Figure 1 – Reference Model of MPAI-AIF Basic Profile

Let’s explore how things can unfold in this new scenario.
1       Creation of infrastructure

Creation of infrastructure is the responsibility of the deployment/control plane to avoid access to the application data plane by the control plane and vice-versa. The REST API protocol is used to specify the steps.

1.1      Connection to the SCI

SCI specifies the required security protocols that the RCA must employ for authentication and authorisation purposes. AIF should include an exemplary list of security protocols (basic, digest, bearer). The connection is required by all subsequent points and must be secured using one of the proposed security schemes described in End Point Open API.

1.2      Creation of an SCI

RCA asks the AIF end point for the creation of one or more SCIs to which all subsequent AIF API requests will be issued. The objective of SCI creation is the acquisition of an SCI identity for use in subsequent API requests to identify the intended SCIs among the many to which the message will be directed.

1.3      Workflow discovery

RCA submits a request to the Server API for AIW matching and discovery. The resulting collection of Workflow Descriptions is returned to the RCA for ultimate selection.

1.4      Launch of the desired AI Workflow

RCA submits a request to the SCI through the AIF end point for the launch of the desired AIW(s). The objective of Workflow launch is the acquisition of a Remote Workflow Instance (RWI) identity for use in subsequent API requests for identification of the intended AIW among the many with which input/output messages will be exchanged.

2       Message Exchange

Application data exchange is the responsibility of the application data plane thus ensuring non-exposure of application data to the control plane. The REST API protocol is used to specify the steps.

2.1      Delivery of messages to the input ports of the AI Workflow

RCA submits requests to the above-identified SCI, through the AIF end point for the delivery of AIF Messages containing application data to the desired input port(s) of the RWI(s).

2.2      Reception of messages from the output ports of the AI Workflow

RCA may submit requests to the above-identified SCI through the AIF end point for the reception of AIF Messages from the desired output port(s) of the launched RWI(s). The RCA makes provision for asynchronous delivery of the response when required.

2.3       Termination of infrastructure

The deployment/control plane is responsible for the avoidance of access to the application data plane by the control plane and vice-versa. The REST API protocol is used to specify the steps.

3    Termination

3.1      Termination of the AI Workflow

RCA submits requests to the SCI through the AIF end point for the termination of the RWI(s).

3.2      Release of the AIF Controller

RCA submits requests to the AIF end point for the termination of the above-identified SCI(s).

Figure 2 depicts an initial Reference Model of MPAI as a Service.

Figure 2 – MPAI as a Service Reference Model

An overview of the complete workflow is given by:

1. The RCA issues a request through the API Client to the API Server for the creation of an SCI.
2. The API Server acts as a local User Agent of a Controller.
3. The API Server returns the ID (created by the API server) of the newly created SCI to RCA.
4. The RCA issues a request via the API Client through the API Server to the indicated SCI for the instantiation of a named AIW (RWI).
5. The SCI retrieves the named AIW metadata (describing the AIW) from the MPAI Store and then parses, retrieves, and installs the referenced packages as required for the instantiation of the AIW.
6. The MPAI Store receives requests from the SCI for delivery of AIW metadata and the subordinate packages that collectively describe the complete AIW.
7. The MPAI Store returns the requested elements if it possesses them, otherwise it issues requests to the appropriate remote repositories so as to retrieve the missing elements. The MPAI Store could be:
   1. As simple as a stand-alone web server responding to HTTP Get requests.
   2. Based on a distributed file system management service, such as HDSF and other variations.
   3. Based on a standard cloud object management and delivery service, such as Amazon S3 or Open Stack Swift.
   4. Fronted by an object authenticity management framework, such as The Update Framework.
   5. Any combination or variation of the above.
8. The API Server returns the AIW ID, which was provided by the SCI to the RCA.
9. The RCA issues a request via the API Client through the API Server to the indicated SCI for delivery of the accompanying input data message to the specified Port of an AIM of the indicated AIW.
10. The RCA issues a request via the API Client through the API Server to the indicated SCI for reception of an output data message from the specified Port of an AIM of the indicated AIW.
11. The API Server returns to the RCA the output data message received from the specified Port of an AIM, which was provided by the indicated SCI.
12. The RCA issues a request via the API Client through the API Server to the indicated SCI for the termination of the RWI.
13. The RCA issues a request via the API Client to the API Server for the termination of the indicated SCI.

To leverage the availability of AIMs and AIWs from various sources, MaaS requires that:

• The access to the MPAI Store be ubiquitous to support envisaged application scenarios.
• The highest level of authorisation be guaranteed by the SCI to an RCA when accessing AI Workflows and their constituent components.
• The highest level of authenticity control be exercised by the SCI on AI Workflows and their constituent packaged components.\

MPAI-66 has issued a Call for Technologies requesting interested parties to propose:

• An architecture for the management of the MPAI Store and the subordinate distributed repositories.
• Protocol(s) that are considered suitable for supporting the above requirements.
• Alternatively, a single interface enabling SCIs to access a plurality of repositories each supporting different protocols.
• If needed, proposals for revision of the MPAI-AIF Basic API, to accommodate requirements of the proposed technologies.

Solutions proposed may be original, or rely on existing technologies, or be any integration thereof.

The MaaS Call will be presented at two online events held on 2026/03/30 at 8:00 UTC (register here to attend) and 15:00 UTC (register here to attend).

The post MPAI as a Service (MaaS) for a new generation of intelligent services appeared first on Leonardo's Blog.

The Health Secure Platform specifies the architecture of a platform offering health-related services enabling the following functionalities:

1. End Users use AIH-HSP Apps running on their Front Ends (personal devices) to acquire Health Data.
2. Health Data, combined with an associated Model Licence, are called AIH Data.
3. AIH Data is uniquely identified.
4. AIH Data is processed by the Front End using an instance of the MPAI-specified AI Framework (MPAI-AIF).
5. Front End processes AIH Data using AI-for-Health-recommended AI Modules (AIM) downloaded from the MPAI Store.
6. Neural Networks in AIMs continually learn while making inferences on AIH Data.
7. Un-processed and Processed AIH Data are uploaded to the AI Back End.
8. Back End stores the Model Licence as a Smart Contract on a Blockchain associated with the Back End.
9. A Smart Contract ID is added to the AIH Data.
10. The Smart Contract governs the use that is made of the AIH Data stored on the Back End.
11. Depending on the relevant Smart Contract, an instance of AIH Data stored on the Back End may be processed by the Back End itself and Third-Party Users.
12. The Back End may process End Users’ AIH Data in its local AI Framework based AI Data Processing AIM.
13. A rich AIH Taxonomy includes:
    1. AIH Data Classes (currently: ECG, EEG, Genomics, and Medical Images).
    2. AIH Data Users (currently: End User, Non-Profit Entity, Profit Entity, Clinical Entity, Authorised Entity, Caregiver).
    3. AIH Data Statuses (currently: Anonymised, Pseudonymised, Identified).
    4. AIH Data Usages (currently: Unrestricted, Pseudonymised, Anonymised, Research, Patient use, Health care).
    5. AIH Data Processing Types (currently: ECG, EEG, Genomics, Medical Images).
    6. Anonymisation/De-Identification Algorithms.
    7. Anomaly Types.

Figure 1 depicts the Health Secure Platform specified by AI for Health. At the centre there is the Back End to which Front Ends and Third-Party Users are connected. The MPAI Store enables Back End and Front Ends to access the AI Modules they need for their processing. The Blockchain manages the licencing terms provided to it by the Model Licence.

Figure 1 – General Model of AIH-HSP V1.0

Figure 2 depicts the architecture of the AIH Back End where Back End, End User, Blockchain, and Third-Party Users perform operations.

Figure 2 – Reference Model of the Health Back End (AIH-HBE) AIW

1. Back End accesses the MPAI Store and downloads the AIMs required for its operation.
2. User Registration
   1. A User wishing to access the Back End, sends a Registration Request containing Personal Profile and list of Service they intend to access.
   2. Back End provides the Tokens enabling the requesting User to access the corresponding Services.
3. Storage of AIH Data
   1. End User uploads AIH Data.
   2. HBE Data Processing
      1. Extracts Model Licence from AIH Data.
      2. Issues Blockchain Licence Request to Blockchain.
   3. Blockchain
      1. Converts Model Licence to a Smart Contract.
      2. Responds with a Blockchain Licence Request.
   4. HBE Data Processing
      1. Attaches Blockchain Licence ID to AIH Data.
      2. Stores AIH Data in Secure Storage
   5. De-Identification/Anonymisation (DIA) of AIH Data
      1. End User sends a DIA Request.
      2. HBE Data Processing
         1. Retrieves relevant AIH Data from Secure Storage.
         2. (Pseudo-)Anonymises AIH Data.
         3. Stores (Pseudo-)Anonymised AIH Data back to Secure Storage.
         4. Responds with a DIA Response.
      3. AIH Data Processing
         1. User sends AIH Process Request.
         2. HBE Data Processing sends a Licence Confirm Request to the Blockchain.
         3. Blockchain responds with a Licence Confirm Response.
         4. HBE Data Processing
            1. Performs the requested Processing, if this is included in the Licence.
            2. Stores the Processed AIH Data as new AIH Data.
            3. Responds with an AI Data Process Response.
         5. Audit
            1. End User sends Audit Request.
            2. Auditing
               1. Retrieves relevant Confirmation Responses to verify that all Processing was performed according to the Licence terms.
               2. Responds with Audit Response.
            3. Federated Learning
               1. Federated Learning sends Federated Learning Request to all Health Front Ends.
               2. Health Front Ends provide the NN Models.
               3. Federated Learning
                  1. Develops and upload the new NN Model to the MPAI Store.
                  2. Sends Federated Learning Response to Health Front Ends.
               4. Front Ends download the new NN Model from the MPAI Store.

Figure 3 depicts the Reference Architecture of the Health Front End (AIH-HFE) where Front End and End User perform operations.

Figure 3 – Reference Model of the Health Front End (AIH-HFE) AIW

1. End User registers with HFE and HBE.
2. End User acquires Health Data with a Health Device and provides Model Licence.
3. Model Licencing AIM attaches Model Licence to Health Data, produces AIH Data and Stores AIH Data.
4. End User processes AIH Data locally.
5. End User stores AIH Data to HFE.
6. End User processes AIH Data remotely on the Back End.
7. HFE receives Federated Learn request.
8. HFE sends the NN Model trained since last Federated Learn request to HBE.

The AIH-HSP V1.0 standard is available. An online presentation will be made on 2026/02/09 T15 UTC. Register to attend.

Comments on AIH-HSP V1.0 shall reach the MPAI Secretariat by 2026/03/16.

The post Improved Health Services with AI appeared first on Leonardo's Blog.

1. A standard for the Autonomous User Architecture
2. A-User Control: The Autonomous Agent’s Brain
3. Context Capture: The A-User’s First Glimpse of the World
4. Audio Spatial Reasoning: The Sound-Aware Interpreter
5. Visual Spatial Reasoning: The Vision‑Aware Interpreter
6. Prompt Creation: Where Words Meet Context
7. Domain Access: The Specialist Brain Plug-in for the Autonomous User
8. Basic Knowledge: The Generalist Engine Getting Sharper with Every Prompt
9. User State Refinement: Turning a Snapshot into a Full Profile
10. Personality Alignment: The Style Engine of A-User
11. A-User Formation: Building the A-User

A standard for the Autonomous User Architecture

MPAI has developed 15 standards to facilitate componentisation of AI applications. One of them is MPAI Metaverse Model – Architecture (MMM-TEC) currently at Version 2.1. MMM-TEC assumes that a Metaverse Instance (M-Instance) is populated by Processes performing Actions on Items either directly or indirectly by requesting another Process to perform Actions on their behalf. The requested Process performs the Action if the requesting and requested Processes have the Rights to do. Process Action is the means for a Process to make requests.

A particularly important Process is the User. This may be driven directly by a human (in which case it is called H-User) or may operate autonomously (in which case it is called A-User) performing Actions and requesting Process Actions.

MMM-TEC provides the technical means for an H-User to act in an M-Instance. An A-User can use the same means to act but it currently does not provide the means to decide what (Process) Actions to perform. Such means are vitally important for an A-User to achieve autonomous agency and thus make M-Instances more attractive places for humans to visit and settle.

After long discussions, MPAI has initiated the Performing Goal in metaverse (MPAI-PGM) project. The first subproject is called Autonomous User Architecture (PGM-AUA). Currently, this includes the following documents: a Call for Technologies (per the MPAI process, a technical standard based on the responses received to a Call) accompanied by Use Cases and Functional Requirements (what the standard is expected to do), Framework Licence (guidelines for the use of Essential IPR of the Standard), and a recommended Template for Responses.

The complexity of the PGM-AUA project has prompted MPAI to develop a Tentative Technical Specification (TTS). This uses the style but is NOT an MPAI Technical Specification. It has been developed as a concrete example of the goal that MPAI intends to eventually achieve with the PGM-AUA project. Respondents to the Call are free to comment on, change, or extend TTS or to propose anything else relevant to the Call whether related or not to the TTS.

Let’s have a look at the Tentative Architecture of Autonomous User.

Anybody is entitled to respond to the Call. Responses shall be submitted to the MPAI Secretariat by 2026/01/21T23:59.

In the following, an extensive techno-conversational description of the TTS is developed.

A-User Control: The Autonomous Agent’s Brain

A-User Control is the general commander of the A-User system making sure the Avatar behaves like a coherent digital entity aware of the rights it can exercise in an instance of the MPAI Metaverse Model – Architecture (MMM-TEC) standard. The command is actuated by various signals exchanged with the Ai-Modules (AIM) composing the Autonomous User.

At its core, A-User Control decides what the A-User should do, which AIM should do it, and how it should do it – all while respecting the Rights held by the A-User in and the Rules defined by the metaverse. Obviously, A-User Control either executes an Action directly or delegates another Process in the metaverse to carry it out.

A-User Control is not just about triggering actions. A-User Control also manages the operation of its AIMs, for instance A-User Formation, which can turn text produced by the Basic Knowledge (LLM) and the Entity Status selected by Personality Alignment into a speaking and gesturing Avatar. A-User Control sends shaping commands to A-User Formation, ensuring the Avatar’s behaviour aligns with metaverse-generated cues and contextual constraints.

A-User Control is not independent of human influence. The human, i.e., the A-User “owner”, can override, adjust, or steer its behaviour. This makes A-User Control a hybrid system: autonomous by design, but open to human modulation when needed.

The control begins when A-User Control triggers Context Capture to perceive the current M-Location – the spatial zone of the metaverse where the User is active. That snapshot, called Context, includes spatial descriptors and a readout of the human’s cognitive and emotional posture called Entity State. From there, the two Spatial Reasoning components – Audio and Visual – use Context to analyse the scene and sending outputs to Domain Access and Prompt Creation, which enrich the User’s input and guide the A-User’s understanding.

As reasoning flows through Basic Knowledge, Domain Access, and User State Refinement, A-User Control ensures that every action, rendering, and modulation is aligned with the A-User’s operational logic.

In summary, the A-User Control is the executive function of the A-User: part orchestrator, part gatekeeper, part interpreter. It’s the reason the Avatar doesn’t just speak – it does so while being aware of the Context – both the spatial and User components – with purpose, permission, and precision.

Context Capture: The A-User’s First Glimpse of the World

Context Capture is the A-User’s sensory front-end – the AIM that opens up perception by scanning the environment and assembling a structured snapshot of what’s out there in the moment. It is the first AI Module (AIM) in the loop providing the data and setting the stage for everything that follows.

When A-User Control decides it’s time to engage, it prompts Context Capture to focus on a specific M-Location – the zone where the User is active, rendering its Avatar.

The product of Context Capture is called Context – a time-stamped, multimodal snapshot that represents the A-User’s initial understanding of the scene. But this isn’t just raw data. Context is composed of two key ingredients: Audio-Visual Scene Descriptors and User State.

The Audio-Visual Scene Descriptors are like a spatial sketch of the environment. They describe what’s visible and audible: objects, surfaces, lighting, motion, sound sources, and spatial layout. They provide the A-User with a sense of “what’s here” and “where things are.” But they’re not perfect. These descriptors are often shallow – they capture geometry and presence but not meaning. A chair might be detected as a rectangular mesh with four legs, but Context Capture doesn’t know if it’s meant to be sat on, moved, or ignored.

That’s where Spatial Reasoning comes in. Spatial Reasoning is the AIM that takes this raw spatial sketch and starts asking the deeper questions:

• “Which object is the User referring to?”
• “Is that sound coming from a relevant source?”
• “Does this object afford interaction, or is it just background?”

It analyses the Context and produces an enhanced Scene Description containing a refined map of spatial relationships, referent resolutions, and interaction constraints and a set of cues that enrich the user’s input – highlighting which objects or sounds are relevant, how close they are, and how they might be used.

These outputs are sent downstream to Domain Access and Prompt Creation. The former refines the spatial understanding of the scene. The latter enriches the A-User’s query when it formulates the prompt to the Basic Knowledge (LLM).

Then there is Entity State – a snapshot of the User’s cognitive, emotional, and attentional posture. Is the User focused, distracted, curious, frustrated? Context Capture reads facial expressions, gaze direction, posture, and vocal tone to infer a baseline state. But again, it’s just a starting point. User behaviour may be nuanced, and initial readings can be incomplete, noisy or ambiguous. That’s why User State Refinement exists – to track changes over time, infer deeper intent, and guide the alignment of the A-User’s expressive behaviour done by Personality Alignment.

In short, Context Capture is the A-User’s first glimpse of the world – a fast, structured perception layer that’s good enough to get started, but not good enough to finish the job. It’s the launchpad for deeper reasoning, richer modulation, and more expressive interaction. Without it, the A-User would be blind. With it, the system becomes situationally aware, emotionally attuned, and ready to reason – but only if the rest of the AIMs do their part.

Audio Spatial Reasoning: The Sound-Aware Interpreter

Audio Spatial Reasoning is the A-User’s acoustic intelligence module – the one that listens, localises, and interprets sound not just as data, but as data having a spatially anchored meaning. Therefore, Its role is not just about “hearing”, it is also about “understanding” where sound is coming from, how relevant it is, and what it implies in the context of the User’s intent in the environment.

When the A-User system receives a Context snapshot from Context Capture – including audio streams with a position and orientation and a description of the User’s emotional state (called Entity State) – Audio Spatial Reasoning start an analysis of directionality, proximity, and semantic importance of incoming sounds. The conclusion is something like “That voice is coming from the left, with a tone of urgence, and its orientation is directed at the A-User.”

All this is represented with an extension of the Audio Scene Descriptors describing:

• Which audio sources are relevant
• Where they are located in 3D space
• How close or far they are
• Whether they’re foreground (e.g., a question) or background (e.g., ambient chatter)

This guide is sent to Prompt Creation and Domain Access. Let’s see what happens with the former. The extended Audio Scene Descriptors are fused with the User’s spoken or written input and the current Entity State. The result is a PC-Prompt – a rich query enriched with text expressing the multimodal information collected so far. This is passed to Basic Knowledge for reasoning.

The Audio Scene Descriptors are further processed and integrated with domain-specific information. The response is called Audio Spatial Directive that includes domain-specific logic, scene priors, and task constraints. For example, if the scene is a medical simulation, Domain Access might tell Audio Spatial Reasoning that “only sounds from authorised personnel should be considered”. This feedback helps Audio Spatial Reasoning refine its interpretation – filtering out irrelevant sounds, boosting priority for critical ones, and aligning its spatial model with the current domain expectations.

Therefore, we can call Audio Spatial Reasoning as the A-User’s auditory guide. It knows where sounds are coming from, what they mean, and how they should influence the A-User’s behaviour. The A-User responds to a sound with spatial awareness, contextual sensitivity, and domain consistency.

Visual Spatial Reasoning: The Vision‑Aware Interpreter

When the A-User acts in a metaverse space, sound doesn’t tell the whole story. The visual scene – objects, zones, gestures, occlusions – is the canvas where situational meaning unfolds. That’s where Visual Spatial Reasoning comes in: it’s the interpreter that makes sense of what the Autonomous User sees, not just what it hears. It can be considered as the visual analyst embedded in the Autonomous User’s “brain” that understands objects’ geometry, relationships, and salience.

Visual Spatial Reasoning doesn’t just list objects; it understands their geometry, relationships, and salience. A chair isn’t just “a chair” – it’s occupied, near a table, partially occluded, or the focus of attention. By enriching raw descriptors into structured semantics, Visual Spatial Reasoning transforms objects made of pixels into actionable targets.

This is what it does

• Scene Structuring: Takes and organises raw visual descriptors into coherent spatial maps.
• Semantic Enrichment: Adds meaning – classifying objects, inferring affordances, and ranking salience.
• Directed Alignment: Filters and prioritises based on the A-User Controller’s intent, ensuring relevance.
• Traceability: Every refinement step is auditable, to trace back why, “that object in the corner” became “the salient target for interaction.”

Without Visual Spatial Reasoning, the metaverse would be a flat stage of unprocessed visuals. With it, visual scenes become interpretable narratives. It’s the difference between “there are three objects in the room” and “the User is focused on the screen, while another entity gestures toward the door.”

Of course, Visual Spatial Reasoning does not replace vision. It bridges the gap between raw descriptors and effective interaction, ensuring that the A‑User can observe, interpret, and act with precision and intent.

If Audio Spatial Reasoning is the metaverse’s “sound‑aware interpreter,” then Visual Spatial Reasoning is its “sight‑aware analyst” that starts by seeing objects and eventually can understand their role, their relevance, and their story in the scene.

Prompt Creation: Where Words Meet Context

The Prompt Creation module is the storyteller and translator in the Autonomous User’s “brain”, It takes raw sensory input  –  audio and visual spatial data of Context (such as objects in a scene with their position, orientation and velocity) and the Entity State – and turns it into a well‑formed prompt that Basic Knowledge can actually understand and respond to.

The audio and visual components of Spatial Reasoning provide the information on things around the User such as “who’s in the room,” “what’s being said,” “what objects are present,” and “what’s the User doing”. Context Capture provides Entity State as a rich description of the A‑User’s understanding of the “internal state” of the User – which may a representation of a biologically real User, if it represents a human, or simulated when the User represents an agent. The task of Prompt Creation is to synthesise these sources of information into a PC‑Prompt Plan. This plan starts from what the User said, adds intent (e.g., “User wants help” or “User is asking a question”), includes the context around the User (e.g., “User is in a virtual kitchen”), and embeds User State (e.g., “User seems confused”).

This information – conveniently represented as a JSON object – is converted into natural language and passed to Basic Knowledge that produces a natural language response called the Initial Response – initial because there are more processing elements in the A‑User pipeline that will refine and improve the answer before it is rendered in the metaverse.

Prompt Creation gives the AI a sense of narrative, so the A-User can:

– Ask the right clarifying question.

– Respond with relevance to the situation.

– Adapt to the environment and User mood.

– Maintain continuity across interactions.

If the User says: “Can you help me cook?”

– Spatial Reasoning notes the User is in a virtual kitchen with utensils and ingredients.

– Entity State suggests the User looks uncertain.

– Prompt Creation combines these into: “User is asking for cooking help, is in a kitchen, seems unsure.”

This Initial Response is then passed to Domain Access, which may elaborate a new prompt enriched with domain-specific information (in this case “cooking”, when Basic Knowledge is not well informed about cooking).

Prompt Creation turns raw multimodal input and spatial information into meaningful prompts so the AI can think, speak, and act with purpose. It is the scriptwriter that ensures the A‑User’s dialogue is not only coherent but also contextually aware, emotionally attuned, and situationally precise.

Domain Access: The Specialist Brain Plug-in for the Autonomous User

The Basic Knowledge module is a generalist language model that “knows a bit of everything.” In contrast, Domain Access is the expert layer that enables the Autonomous User to tap into domain-specific intelligence for deeper understanding of user utterances and their context.

 How Domain Access Works

• Receives Initial Response: Domain Access starts with the response of Basic Knowledge, the generalist model’s response to the prompt generated by Prompt Creation.
• Converts to DA-Input: As the natural language response is not the best way to process the response, it is converted into a JSON object called DA-Input for structured processing.
• Gets specilised knowledge by pulling in domain vocabulary such as, jargon and technical terms.
• Creates the next prompt by using this specialised knowledge:
  • Injects rules and constraints (e.g., standards, legal compliance).
  • Adds reasoning patterns (e.g., diagnostic flows, contractual logic).

All enrichment happens in the JSON domain and so is the produced DA-Prompt Plan – a domain-aware structure ready for conversion into natural language – called DA-Prompt – and resubmission into the knowledge/response pipeline.

Without Domain Access, the A-User is like a clever intern: knowledgeable but lacking depth and experience. With Domain Access, it becomes n experienced professional that can:

• Deliver accurate, context-aware answers.
• Avoid hallucinations by grounding responses in domain rules.
• Address different application domains by swapping or adding domain modules without rebuilding the entire A-User.

Basic Knowledge: The Generalist Engine Getting Sharper with Every Prompt

Basic Knowledge is the core language model of the Autonomous User – the “knows-a-bit-of-everything” brain. It’s the provider of the first response to a prompt but the Autonomous User doesn’t fire off just one answer but four of them in a progressive refinement loop, providing smarter and more context-aware responses with every refined prompt.

The Journey of a Prompt

1. Starts Simple: The first prompt from Prompt Creation is a rough draft because the A-User has only a superficial knowledge of the Context and User intent.
2. Domain Access adds expert seasoning: jargon, compliance rules, reasoning patterns. The prompt becomes richer and sharper.
3. User State Refinement injects dynamic knowledge about the User – refined emotions, more focused goals, better spatial context – so the prompt feels more attuned to what the User feels and wants.
4. Personality Alignment Tells A-User how to Behave: it ensures that the appropriate A-User’s style and mood drive the next prompt.
5. Final Prompt Delivery: when Basic Knowledge receives the last prompt (from Personality Alignment) the final touches have been added.

This sequence of prompts eventually provides:

• Better responses: Each prompt reduces ambiguity.
• Domain grounding: Avoids hallucinations by embedding rules and expert logic.
• Personalisation: Adapts A-User’s tone and content to User State.
• Scalability: Works across domains without retraining.

Basic Knowledge starts as a generalist, but thanks to refined prompts, it ends up delivering expert-level, context-aware, and User-sensitive responses. It starts from a rough sketch and, by iterating with specialist information sources, it provides a final response that includes all the information extracted or produced in the workflow.

 User State Refinement: Turning a Snapshot into a Full Profile

When the A-User begins interacting, it starts with a basic User State captured by Context Capture – location, activity, initial intent, and perhaps a few emotional hints. This initial state is useful, but it’s like a blurry photo: the A-User knows that somebody ps there, but not the details that matter for nuanced interaction.

As the session unfolds, the A-User learns much more thanks to Prompt Creation, Spatial Reasoning, and Domain Access. Suddenly, the A-User understands not just what the User said, but what it meant, the context it operates in, and the reasoning patterns relevant to the domain. This new knowledge is integrated with the initial state so that subsequent steps – especially Personality Alignment and Basic Knowledge – are based on an appropriate understanding of the User State.

Why Update the User State?

Personality Alignment is where the A-User adapts tone, style, and interaction strategy. If it only relies on the first guess of the User State, it risks taking an incongruent attitude – formal when casual is needed, directive when supportive is expected. If the User State can be updated the A-User knows more about:

• The environment incorporating jargon, compliance rules, and reasoning patterns.
• The internal state and can adjust responses to confusion, urgency, or confidence.

The Refinement Process

1. Start with Context Snapshot: capture environment, speech, gestures, and basic emotional cues.
2. Inject Domain Intelligence from Domain Access: technical vocabulary, rules, structured reasoning.
3. Merge New Observations: emotional shifts, spatial changes, updated intent.
4. Validate Consistency: ensure module coherence for reliable downstream use.
5. Feed Forward: pass the refined state to Personality Alignment and sharper prompts to Basic Knowledge.

 Personality Alignment: The Style Engine of A-User

Personality Alignment is where an A-User interacting with a User embedded in a metaverse environment stops being a generic bot and starts acting like a character with intent, tone, and flair. It’s not just a matter of what it utters – it’s about how those words land, how the avatar moves, and how the whole interaction feels.

The figure is an extract from the A-User Architecture Reference model representing Domain Access generating two streams of data related to the User and its environment and two recipient AI Modules: User State Refinement and Personality Alignment.

This is possible because the A-User receives the right inputs driving the Alignment of the A-User Personality with the refined User’s Entity State:

• Personality Context Guide: Domain-specific hints from Domain Access (e.g., “medical setting → professional tone”).
• Expressive State Guide: Emotional and attentional posture of the User (e.g., stressed → calming personality).
• Refined Response: Text from Basic Knowledge in response to User State Refinement prompt.
• Personality Alignment Directive: Commands to tweak or override the personality profile (e.g., “switch to negotiator mode”) from the A-User Control AI Module (AIM).

A smart integration of these inputs enables the A-User to deliver the following outputs:

• A-User Entity State: the complete internal state of the A-User’s synthetic personality produced (tone, gestures, behavioural traits).
• PA-Prompt: New prompt formulation including the final A-User personality (so the words sound right).
• Personality Alignment Status: A structured report of personality and expressive alignment to the A-User Control AIM.

Here are some examples of personality profiles that Personality Alignment could use or blend:

• Mentor Mode: Calm tone, structured answers, moderate gestures, empathy cues.
• Entertainer Mode: Upbeat tone, humour, wide gestures, animated expressions.
• Negotiator Mode: Firm tone, controlled gestures, strategic phrasing.
• Assistant Mode: Neutral tone, minimal gestures, clarity-first responses.

A-User Formation: Building the A-User

If Personality Alignment gives the A-User its style, A-User Formation AIM gives the A-User its body and its voice, the avatar and the speech for the A-User Control to embed in the metaverse. The A-User stops being an abstract brain controlling various types of processing and becomes a visible, interactive entity. It’s not just about projecting a face on a bot; it’s about creating a coherent representation that matches the personality, the context, and the expressive cues.

Here is how this is achieved.

Inputs Driving A-User Formation:

• A-User Entity Status: The personality blueprint from Personality Alignment (tone, gestures, behavioural traits).
• Final Response: personality-tuned content from Basic Knowledge – what the avatar will utter.
• A-User Control Command: Directives for rendering and positioning in the metaverse (e.g., MM-Add, MM-Move).
• Rendering Parameters: Synchronisation cues for speech, facial expressions, and gestures.

What comes out of the box is a multimodal representation of the A-User (Speaking Avatar) that talks, moves, and reacts in sync with the A-User’s intent – the best expression the A-User can give of itself in the circumstances.

What Makes A-User Formation Special?

It’s the last mile of the pipeline – the point where all upstream intelligence (context, reasoning, User’s Entity Status estimation, personality) becomes visible and interactive. A-User Formation ensures:

• Expressive Coherence: Speech, gestures, and facial cues match the chosen personality.
• Contextual Fit: Avatar appearance and behaviour align with domain norms (e.g., formal in a medical setting, casual in a social lounge).
• Technical Precision: Synchronisation across Personal Status modalities for natural and consistent interaction.
• Goal: Deliver a coherent, expressive, and context-aware representation that feels natural and engaging in response to how the User was perceived at the beginning and processed during the pipeline.

The post A Walk inside the Autonomous User appeared first on Leonardo's Blog.

We have already presented the system diagram of the Autonomous User (A-User), an autonomous agent able to move and interact (walk, converse, do things, etc.) with another User in a metaverse. The latter User may be an A-User or be under the direct control of a human and is thus called a Human-User (H-User). The A-User acts as a “conversation partner in a metaverse interaction” with the User.

This is the tenth and last of a sequence of posts aiming to illustrate more in depth the architecture of an A-User and provide an easy entry point for those who wish to respond to the MPAI Call for Technology on Autonomous User Architecture. The first six dealt with 1) the Control performed by the A-User Control AI Module on the other components of the A-User; 2) how the A-User captures the external metaverse environment using the Context Capture AI Module; 3) listens, localises, and interprets sound not just as data, but as data having a spatially anchored meaning; 4) makes sense of what the Autonomous User sees by understanding objects’ geometry; relationships, and salience; 5) takes raw sensory input and the User State and turns them into a well‑formed prompt that Basic Knowledge can actually understand and respond to; 6) taps into domain-specific intelligence for deeper understanding of user utterances and operational context; 7) the core language model of the Autonomous User – the “knows-a-bit-of-everything” brain, the first responder to a prompt of a sequence of four; 8) converting a “blurry photo” of the User in the environment taken at the onset of the process into a focused picture; and 9) providing not only a generic bot but a character with intent, tone, and flair – not only a matter of what the avatar utters but how its words land, how the avatar moves, and how the whole interaction feels.

 A-User Formation AIM gives the A-User a body and a voice, the results of a chain or AI Modules composing the A-User pipeline enabling a perceptible and coherent representation that matches the personality, the context, and the expressive cues.

Here is how this is achieved.

Inputs Driving A-User Formation

• A-User Entity Status: The personality blueprint from Personality Alignment (tone, gestures, behavioural traits).
• Final Response: personality-tuned content from Basic Knowledge – what the avatar will utter.
• A-User Control Command: Directives for rendering and positioning in the metaverse (e.g., MM-Add, MM-Move).
• Rendering Parameters: Synchronisation cues for speech, facial expressions, and gestures.

What comes out of the box: Formation Status

• A multimodal representation of the A-User (Speaking Avatar) that talks, moves, and reacts in sync with the A-User’s intent – the best expression the A-User can give of itself in the circumstances.
• Structured report on the processing that led to the result.

What Makes A-User Formation Special?

It’s the last mile of the pipeline – the point where all upstream intelligence (context, reasoning, User’s Entity Status estimation, personality) becomes visible and interactive. A-User Formation ensures:

• Expressive Coherence: Speech, gestures, and facial cues match the chosen personality.
• Contextual Fit: Avatar appearance and behaviour align with domain norms (e.g., formal in a medical setting, casual in a social lounge).
• Technical Precision: Synchronisation across Personal Status modalities for natural and consistent interaction.

Key Points to Take Away about A-User Formation

1. Purpose: Turns the A-User’s personality and reasoning into a visible and audible interactive avatar.
2. Inputs: Personality-aligned final response, control commands, and rendering parameters.
3. Outputs: Speaking avatar, formation status.
4. Goal: Deliver a coherent, expressive, and context-aware representation that feels natural and engaging in response to how the User was perceived at the beginning and processed during the pipeline.

The post A-User Formation: Building the A-User appeared first on Leonardo's Blog.

We have already presented the system diagram of the Autonomous User (A-User), an autonomous agent able to move and interact (walk, converse, do things, etc.) with another User in a metaverse. The latter User may be an A-User or be under the direct control of a human and is thus called a Human-User (H-User). The A-User acts as a “conversation partner in a metaverse interaction” with the User.

This is the tenth and last of a sequence of posts aiming to illustrate more in depth the architecture of an A-User and provide an easy entry point for those who wish to respond to the MPAI Call for Technology on Autonomous User Architecture. The first six dealt with 1) the Control performed by the A-User Control AI Module on the other components of the A-User; 2) how the A-User captures the external metaverse environment using the Context Capture AI Module; 3) listens, localises, and interprets sound not just as data, but as data having a spatially anchored meaning; 4) makes sense of what the Autonomous User sees by understanding objects’ geometry; relationships, and salience; 5) takes raw sensory input and the User State and turns them into a well‑formed prompt that Basic Knowledge can actually understand and respond to; 6) taps into domain-specific intelligence for deeper understanding of user utterances and operational context; 7) the core language model of the Autonomous User – the “knows-a-bit-of-everything” brain, the first responder to a prompt of a sequence of four; 8) converting a “blurry photo” of the User in the environment taken at the onset of the process into a focused picture; and 9) providing not only a generic bot but a character with intent, tone, and flair – not only a matter of what the avatar utters but how its words land, how the avatar moves, and how the whole interaction feels.

A-User Formation AIM gives the A-User a body and a voice, the results of a chain or AI Modules composing the A-User pipeline enabling a perceptible and coherent representation that matches the personality, the context, and the expressive cues.

The inputs driving A-User Formation are

• A-User Entity Status: The personality blueprint from Personality Alignment (tone, gestures, behavioural traits).
• Final Response: personality-tuned content from Basic Knowledge – what the avatar will utter.
• A-User Control Command: Directives for rendering and positioning in the metaverse (e.g., MM-Add, MM-Move).
• Rendering Parameters: Synchronisation cues for speech, facial expressions, and gestures.

What comes out of the box: Formation Status

• A multimodal representation of the A-User (Speaking Avatar) that talks, moves, and reacts in sync with the A-User’s intent – the best expression the A-User can give of itself in the circumstances.
• Structured report on the processing that led to the result.

What Makes A-User Formation Special?

It’s the last mile of the pipeline – the point where all upstream intelligence (context, reasoning, User’s Entity Status estimation, personality) becomes visible and interactive. A-User Formation ensures:

• Expressive Coherence: Speech, gestures, and facial cues match the chosen personality.
• Contextual Fit: Avatar appearance and behaviour align with domain norms (e.g., formal in a medical setting, casual in a social lounge).
• Technical Precision: Synchronisation across Personal Status modalities for natural and consistent interaction.

Key Points to Take Away about A-User Formation

1. Purpose: Turns the A-User’s personality and reasoning into a visible and audible interactive avatar.
2. Inputs: Personality-aligned final response, control commands, and rendering parameters.
3. Outputs: Speaking avatar, formation status.
4. Goal: Deliver a coherent, expressive, and context-aware representation that feels natural and engaging in response to how the User was perceived at the beginning and processed during the pipeline.

The post A-User Formation: Building the A-User appeared first on Leonardo's Blog.

We have already presented the system diagram of the Autonomous User (A-User), an autonomous agent able to move and interact (walk, converse, do things, etc.) with another User in a metaverse. The latter User may be an A-User or be under the direct control of a human and is thus called a Human-User (H-User). The A-User acts as a “conversation partner in a metaverse interaction” with the User.

This is the ninth of a sequence of posts aiming to illustrate more in depth the architecture of an A-User and provide an easy entry point for those who wish to respond to the MPAI Call for Technology on Autonomous User Architecture. The first six dealt with 1) the Control performed by the A-User Control AI Module on the other components of the A-User; 2) how the A-User captures the external metaverse environment using the Context Capture AI Module; 3) listens, localises, and interprets sound not just as data, but as data having a spatially anchored meaning; 4) makes sense of what the Autonomous User sees by understanding objects’ geometry; relationships, and salience; 5) takes raw sensory input and the User State and turns them into a well‑formed prompt that Basic Knowledge can actually understand and respond to; 6) taps into domain-specific intelligence for deeper understanding of user utterances and operational context; 7) the core language model of the Autonomous User – the “knows-a-bit-of-everything” brain, the first responder to a prompt of a sequence of four; and 8) converting a “blurry photo” of the User in the environment taken at the onset of the process into a focused picture.

The figure is an extract from the A-User Architecture Reference model representing Domain Access generating two streams of data related to the User and its environment and two recipient AI Modules: User State Refinement and Personality Alignment.

This is possible because the A-User receives the right inputs driving the Alignment of the A-User Personality with the refined User’s Entity State:

• Personality Context Guide: Domain-specific hints from Domain Access (e.g., “medical setting → professional tone”).
• Expressive State Guide: Emotional and attentional posture of the User (e.g., stressed → calming personality).
• Refined Response: Text from Basic Knowledge in response to User State Refinement prompt.
• Personality Alignment Directive: Commands to tweak or override the personality profile (e.g., “switch to negotiator mode”) from the A-User Control AI Module (AIM).

A smart integration of these inputs enables the A-User to deliver the following outputs:

• A-User Entity State: the complete internal state of the A-User’s synthetic personality produced (tone, gestures, behavioural traits).
• PA-Prompt: New prompt formulation including the final A-User personality (so the words sound right).
• Personality Alignment Status: A structured report of personality and expressive alignment to the A-User Control AIM.

Here are some examples of personality profiles that Personality Alignment could use or blend:

• Mentor Mode: Calm tone, structured answers, moderate gestures, empathy cues.
• Entertainer Mode: Upbeat tone, humour, wide gestures, animated expressions.
• Negotiator Mode: Firm tone, controlled gestures, strategic phrasing.
• Assistant Mode: Neutral tone, minimal gestures, clarity-first responses.

Key Points to Take Away about Personality Alignment

• Purpose: Makes A-User’s delivery context-aware and emotionally tuned.
• Inputs: Domain context, user emotional state, refined semantic response, and directives.
• Outputs: Personality blueprint (Entity Status), PA-Prompt for expressive rendering, and alignment status.
• Profiles: For example, Mentor, Entertainer, Negotiator, Assistant – each with tone, gesture style, and behavioural traits.
• Goal: Coherent, adaptive interaction that feels natural and persuasive in the metaverse.

The post Personality Alignment: The Style Engine of A-User appeared first on Leonardo's Blog.