94f74f2dfc
- Architecture globale (React Native + NestJS + FastAPI) - Beckn Protocol (OTN-DPI) integration docs - AI layer: RAG pipeline + AI Agents (LocalAI + Qdrant) - i18n: FR/EN/ES/DE support - Docker Compose for backend services - Project structure with frontend, backend, ai, beckn directories
222 lines
9.6 KiB
Markdown
222 lines
9.6 KiB
Markdown
# Smart App City — AI Architecture
|
|
|
|
## Overview
|
|
AI-powered features for the Smart App City mobile application, leveraging RAG, LLM, and AI Agents.
|
|
|
|
## AI Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ AI Pipeline │
|
|
│ │
|
|
│ ┌───────────────────────────────────────────────────────────┐ │
|
|
│ │ User Query │ │
|
|
│ │ "Quel est le bus pour Fort-de-France à 14h?" │ │
|
|
│ └───────────────────────┬───────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌───────────────────────────────────────────────────────────┐ │
|
|
│ │ NLP / Intent Classifier │ │
|
|
│ │ - Intent: transport.search │ │
|
|
│ │ - Entities: {time: "14h", destination: "Fort-de-France"}│ │
|
|
│ │ - Language: fr │ │
|
|
│ └───────────────────────┬───────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ┌───────────┼───────────┐ │
|
|
│ ▼ ▼ ▼ │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ Transport │ │ Tourisme │ │ Services │ │
|
|
│ │ Agent │ │ Agent │ │ Agent │ │
|
|
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
|
|
│ │ │ │ │
|
|
│ └────────────────┴────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌───────────────────────────────────────────────────────────┐ │
|
|
│ │ RAG Pipeline │ │
|
|
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
|
|
│ │ │ Query │ │ Embed │ │ Vector │ │ Rerank │ │ │
|
|
│ │ │ Rewrite │ │ (E5) │ │ Search │ │ (BM25+) │ │ │
|
|
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
|
|
│ └───────────────────────┬───────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌───────────────────────────────────────────────────────────┐ │
|
|
│ │ LLM Generation │ │
|
|
│ │ - Context: Retrieved documents + real-time data │ │
|
|
│ │ - Model: Llama 3.1 70B (LocalAI) │ │
|
|
│ │ - Output: Natural language response │ │
|
|
│ └───────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Components
|
|
|
|
### 1. LocalAI (LLM Server)
|
|
- **Container**: `localai-api` (existing)
|
|
- **Model**: Llama 3.1 70B quantized or Mistral 7B
|
|
- **API**: OpenAI-compatible (`/v1/completions`, `/v1/chat/completions`)
|
|
- **Endpoint**: `http://localai-api:8080`
|
|
|
|
### 2. RAG Pipeline
|
|
|
|
#### Embedding
|
|
- **Model**: `intfloat/multilingual-e5-large` (multilingual)
|
|
- **Dimensions**: 1024
|
|
- **Input**: Documents, FAQ, municipal data, tourist guides
|
|
|
|
#### Vector Database
|
|
- **Engine**: Qdrant (existing)
|
|
- **Collection**: `smart-city-docs`
|
|
- **Payload**: `{doc_type, language, timestamp, source}`
|
|
|
|
#### Documents to Index
|
|
| Source | Type | Language |
|
|
|--------|------|----------|
|
|
| Municipal regulations | PDF/HTML | FR, EN |
|
|
| Tourist guides | Markdown | FR, EN, ES, DE |
|
|
| Transport schedules | JSON/API | FR |
|
|
| FAQ | Markdown | FR, EN, ES, DE |
|
|
| Event calendar | API | FR |
|
|
| Sensor data descriptions | JSON | FR, EN |
|
|
|
|
### 3. AI Agents
|
|
|
|
#### Transport Agent
|
|
- **Tools**: Bus API, Parking API, Traffic API
|
|
- **Capabilities**: Schedule lookup, route planning, real-time delays
|
|
- **Data Sources**: FROST-Server, Orion-LD
|
|
|
|
#### Tourism Agent
|
|
- **Tools**: POI database, Weather API, Event API
|
|
- **Capabilities**: Recommendations, itineraries, booking suggestions
|
|
- **Data Sources**: PostGIS (POI), OpenWeatherMap
|
|
|
|
#### Services Agent
|
|
- **Tools**: Forms API, Appointment API, Document API
|
|
- **Capabilities**: Procedure guidance, appointment scheduling
|
|
- **Data Sources**: City ERP
|
|
|
|
#### Environment Agent
|
|
- **Tools**: Sensor API, Air Quality API, Weather API
|
|
- **Capabilities**: Real-time monitoring, health recommendations
|
|
- **Data Sources**: InfluxDB (IoT sensors)
|
|
|
|
## Implementation
|
|
|
|
### RAG Service (Python/FastAPI)
|
|
```python
|
|
# ai/rag-service/main.py
|
|
from fastapi import FastAPI
|
|
from langchain import hub
|
|
from langchain_community.embeddings import HuggingFaceEmbeddings
|
|
from langchain_community.vectorstores import Qdrant
|
|
from langchain_community.llms import OpenAI
|
|
|
|
app = FastAPI()
|
|
|
|
# Connect to LocalAI
|
|
llm = OpenAI(
|
|
base_url="http://localai-api:8080/v1",
|
|
api_key="dummy",
|
|
model="llama-3.1-70b"
|
|
)
|
|
|
|
# Connect to Qdrant
|
|
embeddings = HuggingFaceEmbeddings(
|
|
model_name="intfloat/multilingual-e5-large"
|
|
)
|
|
vectorstore = Qdrant.from_existing_collection(
|
|
collection_name="smart-city-docs",
|
|
embedding=embeddings,
|
|
url="http://qdrant:6333"
|
|
)
|
|
|
|
@app.post("/query")
|
|
async def query(request: QueryRequest):
|
|
# Vector search
|
|
docs = vectorstore.similarity_search(request.query, k=5)
|
|
|
|
# RAG prompt
|
|
prompt = hub.pull("rlm/rag-prompt")
|
|
chain = prompt | llm | StrParser()
|
|
|
|
response = chain.invoke({
|
|
"question": request.query,
|
|
"context": format_docs(docs)
|
|
})
|
|
|
|
return {"response": response, "sources": [d.metadata for d in docs]}
|
|
```
|
|
|
|
### Agent Service (Python/LangChain)
|
|
```python
|
|
# ai/agent-service/agents/transport.py
|
|
from langchain.agents import AgentExecutor, create_openai_functions_agent
|
|
from langchain_community.tools import Tool
|
|
|
|
tools = [
|
|
Tool(
|
|
name="bus_schedule",
|
|
func=bus_schedule_lookup,
|
|
description="Lookup bus schedules. Input: route_id, time"
|
|
),
|
|
Tool(
|
|
name="parking_availability",
|
|
func=parking_lookup,
|
|
description="Check parking availability. Input: location, time"
|
|
),
|
|
Tool(
|
|
name="traffic_conditions",
|
|
func=traffic_lookup,
|
|
description="Get traffic conditions. Input: route, time"
|
|
)
|
|
]
|
|
|
|
agent = create_openai_functions_agent(llm, tools, prompt)
|
|
executor = AgentExecutor(agent=agent, tools=tools)
|
|
```
|
|
|
|
## Multilingual Support
|
|
|
|
### LLM Prompts per Language
|
|
```yaml
|
|
# ai/agent-service/prompts/fr.yml
|
|
transport_agent: |
|
|
Tu es un assistant transport pour la Martinique.
|
|
Réponds en français de manière concise et utile.
|
|
Donne les horaires de bus, places de parking, et conditions de circulation.
|
|
|
|
# ai/agent-service/prompts/en.yml
|
|
transport_agent: |
|
|
You are a transport assistant for Martinique.
|
|
Respond in English concisely and helpfully.
|
|
Provide bus schedules, parking availability, and traffic conditions.
|
|
```
|
|
|
|
### Translation Fallback
|
|
For languages without specific prompts, use automatic translation:
|
|
1. Detect language → translate query to FR/EN
|
|
2. Process in FR/EN
|
|
3. Translate response back to original language
|
|
|
|
## Roadmap
|
|
|
|
### Phase 1: RAG Setup
|
|
- [ ] Deploy Qdrant collection
|
|
- [ ] Index municipal documents
|
|
- [ ] Connect LocalAI
|
|
- [ ] Basic Q&A endpoint
|
|
|
|
### Phase 2: AI Agents
|
|
- [ ] Transport agent
|
|
- [ ] Tourism agent
|
|
- [ ] Services agent
|
|
- [ ] Environment agent
|
|
|
|
### Phase 3: Advanced Features
|
|
- [ ] Multi-turn conversations
|
|
- [ ] Personalization
|
|
- [ ] Predictive recommendations
|
|
- [ ] Voice interface (STT/TTS)
|