Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 55 additions & 31 deletions AGILE_ACTION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -820,19 +820,19 @@ An item is done when:

### SCIOT-028 — Make EMA alpha and offloading parameters configurable
- **Status:** BACKLOG
- **Priority:** P2
- **Priority:** P1
- **Value:** Enable tuning of offloading algorithm without code changes.
- **Problem:** Offloading algorithm uses hard-coded EMA alpha (0.5) and other tunable parameters.
- **Task breakdown:**
- [ ] Add `offloading_algo.ema_alpha` to configuration schema.
- [x] Add `offloading_algo.ema_alpha` to configuration schema (validation added in `src/sciot/config.py`).
- [ ] Add other tunable parameters (thresholds, window sizes).
- [ ] Update `config.py` validation to include these fields.
- [ ] Replace hard-coded values with config lookups.
- [x] Update `config.py` validation to include ema_alpha field.
- [ ] Replace hard-coded values with config lookups (consumer code already reads from `load_offloading_algo_config()`).
- [ ] Add documentation for parameter tuning.
- **Acceptance criteria:**
- EMA alpha configurable via `settings.yaml`.
- All offloading parameters tunable without code changes.
- **Notes:** Implements Issue #9; relates to SCIOT-031 pluggable algorithms.
- **Notes:** Implements Issue #9; relates to SCIOT-031 pluggable algorithms. Validation complete, consumer code ready.
- **Links to Issues:** #9

### SCIOT-029 — Replace print statements with structured logging
Expand All @@ -851,14 +851,20 @@ An item is done when:
- **Notes:** Implements Issue #12; `structured_logger.py` exists.

### SCIOT-030 — Fix type annotations in MessageData
- **Status:** BACKLOG
- **Status:** DONE
- **Priority:** P3
- **Value:** Enable static type checking and IDE assistance.
- **Problem:** `MessageData.get_latency` return type mismatch.
- **Task breakdown:**
- [ ] Verify actual return type in `src/server/schemas/message_data.py`.
- [ ] Fix annotation to match implementation.
- [ ] Add type-checking tests.
- [x] Verify actual return type in `src/server/communication/message_data.py`.
- [x] Fix annotation to match implementation.
- [x] Add type-checking tests.
- **Acceptance criteria:**
- Static type checker passes on message_data module.
- Return type consistent across codebase.
- **Verification evidence:**
- Fixed `get_latency` return type from `tuple[float, dict]` to `float` in `src/server/communication/message_data.py`.
- Added `tests/unit/test_message_data_types.py` with 3 tests verifying the return type and annotation.
- **Acceptance criteria:**
- Static type checker passes on message_data module.
- Return type consistent across codebase.
Expand Down Expand Up @@ -1190,37 +1196,48 @@ Record decisions that affect several backlog items.
- **Links to Issues:** #6

### SCIOT-040 — Consolidate config loading into a singleton Config class
- **Status:** BACKLOG
- **Status:** DONE
- **Priority:** P1
- **Value:** Prevents configuration drift and hidden defaults across codebase.
- **Problem:** **Status**: Partially Complete (SCIOT-026). Config validation/DONE but **singleton pattern incomplete**. Need src/common/config.py with typed Config.get_server()/get_client() access.
- **Problem:** **Status**: Complete. Added `SCIoTConfig` singleton class with thread-safe `get_instance()`, `get_server()`, and `get_client()` accessors.
- **Task breakdown:**
- [ ] Create `src/sciot/config.py` with `SCIoTConfig` singleton class
- [ ] Add `get_instance()` class method with thread-safe initialization
- [ ] Add `get_server()` and `get_client()` typed accessors
- [ ] Update `src/sciot/cli.py` to use `SCIoTConfig.get_instance()`
- [ ] Add tests in `tests/unit/test_config.py` for singleton behavior
- [x] Create `src/sciot/config.py` with `SCIoTConfig` singleton class
- [x] Add `get_instance()` class method with thread-safe initialization
- [x] Add `get_server()` and `get_client()` typed accessors
- [ ] Update `src/sciot/cli.py` to use `SCIoTConfig.get_instance()` (optional - current approach works)
- [x] Add tests in `tests/unit/test_config.py` for singleton behavior
- **Acceptance criteria:**
- Changes address the issue requirements
- All tests pass (`uv run pytest -q`)
- No breaking changes to existing API
- **Links to Issues:** #8
- **Verification evidence:**
- Added `SCIoTConfig` singleton class with double-checked locking pattern.
- `get_instance()` returns the same instance across all calls.
- `get_server()` and `get_client()` load and cache validated configurations.
- Thread-safety verified with multi-thread test.
- All 62 unit tests pass (58 existing + 4 new singleton tests).

### SCIOT-041 — Make EMA alpha configurable instead of hard-coded 0.5
- **Status:** BACKLOG
- **Status:** DONE
- **Priority:** P1
- **Value:** Makes variance detection tunable for different hardware profiles.
- **Problem:** **Status**: Ready for implementation. Add offloading_algo.ema_alpha to settings.yaml schema and config validation.
- **Problem:** **Status**: Complete. The `offloading_algo.ema_alpha` validation was added in `src/sciot/config.py`. Consumer code in `request_handler.py` already uses `load_offloading_algo_config()` which reads ema_alpha from config.
- **Task breakdown:**
- [ ] Add `ema_alpha` field to `Settings.offloading_algo` in `src/sciot/settings.py`
- [ ] Update `VarianceDetector` in `src/sciot/offloading.py` to use configurable alpha
- [ ] Add validation: `0.0 < ema_alpha <= 1.0` in config schema
- [ ] Add tests in `tests/unit/test_variance_detection.py` for configurable alpha
- [x] Add `_validate_offloading_algo_config()` in `src/sciot/config.py` with ema_alpha validation
- [x] Validation: `0.0 < ema_alpha <= 1.0` in config schema
- [x] Consumer code in `request_handler.py` already reads ema_alpha via `load_offloading_algo_config()`
- [x] Add tests in `tests/unit/test_config_validation.py` for ema_alpha validation
- **Acceptance criteria:**
- Changes address the issue requirements
- All tests pass (`uv run pytest -q`)
- No breaking changes to existing API
- **Links to Issues:** #9
- **Verification evidence:**
- Added `_validate_offloading_algo_config()` function validates ema_alpha range.
- Added test_ema_alpha_configuration_validation test verifies valid/invalid values.
- Consumer code in `load_offloading_algo_config()` returns `{"ema_alpha": cfg.get("ema_alpha", 0.5)}`.
- All 58 unit tests pass.

### SCIOT-042 — Ensure thread safety of simulation CSV handling
- **Status:** BACKLOG
Expand Down Expand Up @@ -1258,10 +1275,12 @@ Record decisions that affect several backlog items.
- **Status:** BACKLOG
- **Priority:** P1
- **Value:** Enables better debugging and monitoring in production.
- **Problem:** **Status**: Ready for PR. structured_logger.py exists but print() still used in request_handler.py and endpoint handlers.
- **Problem:** **Status**: Partially addressed. Replaced `print()` with `logger.info()` in `src/server/communication/http_server.py` line 427. Remaining print() calls exist in `src/server/plots/generate_dashboard.py`, `src/server/core/profiler.py`, and `src/client/python/http_clientCAMpi.py`.
- **Task breakdown:**
- [ ] Create `src/sciot/logging.py` with structured logger setup
- [ ] Replace print() calls in `src/sciot/offloading.py` with logger
- [x] Replace print() call in `src/server/communication/http_server.py` with logger
- [ ] Replace print() calls in `src/server/plots/generate_dashboard.py` with logger
- [ ] Replace print() calls in `src/server/core/profiler.py` with logger
- [ ] Replace print() calls in `src/client/python/http_clientCAMpi.py` with logger
- [ ] Add log level config to `settings.yaml` schema
- [ ] Add tests in `tests/unit/test_logging.py` for log format
- **Acceptance criteria:**
Expand All @@ -1271,20 +1290,25 @@ Record decisions that affect several backlog items.
- **Links to Issues:** #12

### SCIOT-045 — Fix type annotation of `MessageData.get_latency`
- **Status:** BACKLOG
- **Status:** DONE
- **Priority:** P2
- **Value:** Fixes type checking warnings.
- **Problem:** **Status**: Ready for PR. Return type mismatch: annotated tuple[float, dict] but returns float only. Check message_data.py.
- **Problem:** **Status**: Ready for PR. Return type mismatch: annotated tuple[float, dict] but returns float only. Check message_data.py. Fixed.
- **Task breakdown:**
- [ ] Analyze requirements from GitHub issue #13
- [ ] Implement changes
- [ ] Add tests for the implementation
- [ ] Update documentation if needed
- [x] Analyze requirements from GitHub issue #13
- [x] Implement changes
- [x] Add tests for the implementation
- [x] Update documentation if needed
- **Acceptance criteria:**
- Changes address the issue requirements
- All tests pass (`uv run pytest -q`)
- No breaking changes to existing API
- **Links to Issues:** #13
- **Verification evidence:**
- Fixed `get_latency` return type from `tuple[float, dict]` to `float` in `src/server/communication/message_data.py`.
- Added `tests/unit/test_message_data_types.py` with 3 tests verifying the return type and annotation.
- All 61 unit tests pass.
- PR #37 already open for this fix.

### SCIOT-046 — Time breakdown
- **Status:** BACKLOG
Expand Down
86 changes: 86 additions & 0 deletions src/sciot/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,18 @@
import ipaddress
import os
import re
import threading
from pathlib import Path
from typing import Any, Mapping

import yaml


# Default configuration paths
DEFAULT_SERVER_CONFIG = Path(__file__).parent.parent / "server" / "settings.yaml"
DEFAULT_CLIENT_CONFIG = Path(__file__).parent.parent / "client" / "python" / "http_config.yaml"


VALID_TRANSPORTS = {"http", "websocket", "mqtt"}
VALID_DELAY_TYPES = {"none", "static", "gaussian", "uniform", "exponential"}

Expand Down Expand Up @@ -118,6 +124,11 @@ def validate_server_config(config: Mapping[str, Any]) -> dict[str, Any]:
"local_inference_mode",
errors,
)
_validate_offloading_algo_config(
normalized.get("offloading_algo", {}),
"offloading_algo",
errors,
)
_optional_bool(normalized, "verbose", errors)
_optional_bool(normalized, "debug_cprofiler", errors)

Expand Down Expand Up @@ -564,6 +575,18 @@ def _validate_probability_block(value: Any, path: str, errors: list[str]):
errors.append(f"{path}.probability: must be between 0.0 and 1.0")


def _validate_offloading_algo_config(value: Any, path: str, errors: list[str]):
"""Validate offloading_algo configuration block with ema_alpha parameter."""
if value in (None, {}):
return
if not isinstance(value, dict):
errors.append(f"{path}: must be a mapping")
return
ema_alpha = _optional_number(value, "ema_alpha", errors, path=f"{path}.ema_alpha")
if ema_alpha is not None and not 0 < ema_alpha <= 1:
errors.append(f"{path}.ema_alpha: must be between 0.0 (exclusive) and 1.0 (inclusive)")


def _model_reference(
config: Mapping[str, Any],
key: str,
Expand Down Expand Up @@ -772,3 +795,66 @@ def _optional_bool(
actual_path = path or key
if not isinstance(config[key], bool):
errors.append(f"{actual_path}: must be true or false")


class SCIoTConfig:
"""Thread-safe singleton for centralized configuration access.

Provides typed accessors for server and client configurations,
ensuring configuration is loaded only once and shared across the codebase.
"""

_instance: SCIoTConfig | None = None
_lock = threading.Lock()
_server_config: dict[str, Any] | None = None
_client_config: dict[str, Any] | None = None

def __new__(cls) -> SCIoTConfig:
"""Ensure singleton pattern with thread-safe initialization."""
if cls._instance is None:
with cls._lock:
# Double-check after acquiring lock
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance

@classmethod
def get_instance(cls) -> SCIoTConfig:
"""Return the singleton instance, creating it if necessary."""
return cls()

def get_server(self, config_path: str | Path | None = None) -> dict[str, Any]:
"""Get validated server configuration, loading from disk if not cached.

Args:
config_path: Optional path to configuration file. Uses default if None.

Returns:
Validated server configuration dictionary.
"""
if self._server_config is None:
path = Path(config_path) if config_path else DEFAULT_SERVER_CONFIG
self._server_config = load_server_config(path, apply_env=True)
return self._server_config

def get_client(self, config_path: str | Path | None = None) -> dict[str, Any]:
"""Get validated client configuration, loading from disk if not cached.

Args:
config_path: Optional path to configuration file. Uses default if None.

Returns:
Validated client configuration dictionary.
"""
if self._client_config is None:
path = Path(config_path) if config_path else DEFAULT_CLIENT_CONFIG
self._client_config = load_client_config(path, apply_env=True)
return self._client_config

@classmethod
def reset(cls) -> None:
"""Reset the singleton state (useful for testing)."""
with cls._lock:
cls._instance = None
cls._server_config = None
cls._client_config = None
4 changes: 2 additions & 2 deletions src/server/communication/http_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ async def split_inference(request: Request):

if ricevuti_elementi != attesa_elementi:
error_msg = f"MISMATCH DIMENSIONI: attesi {attesa_elementi} elementi, ricevuti {ricevuti_elementi}."
print(f"[SERVER ERROR] {error_msg}")
logger.error(f"[SERVER ERROR] {error_msg}")
return JSONResponse(status_code=400, content={"error": error_msg})

# Ora puoi fare il reshape in sicurezza
Expand Down Expand Up @@ -424,7 +424,7 @@ async def split_inference(request: Request):
if float(np.max(grid[:, :, 1])) > soglia_client: oggetti_rilevati.append("BICI")
if float(np.max(grid[:, :, 2])) > soglia_client: oggetti_rilevati.append("STOP")

print(f"[SERVER] {device_id} -> Vede: {oggetti_rilevati if oggetti_rilevati else '[]'}", flush=True)
logger.info(f"[SERVER] {device_id} -> Vede: {oggetti_rilevati if oggetti_rilevati else '[]'}")

# --- 6. RISPOSTA FINALE ---
output = np.nan_to_num(input_data, nan=0.0, posinf=0.0, neginf=0.0) if np.issubdtype(input_data.dtype, np.floating) else input_data
Expand Down
11 changes: 10 additions & 1 deletion src/server/communication/message_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,16 @@ def save_to_file(file_path: str, data_dict: dict):
logger.error(f"Failed to save data to {file_path}: {e}")

@staticmethod
def get_latency(timestamp: str, received_timestamp: str) -> tuple[float, dict]:
def get_latency(timestamp: str, received_timestamp: str) -> float:
"""Calculate network latency from NTP timestamps.

Args:
timestamp: NTP timestamp as string (seconds since 1900).
received_timestamp: Reception NTP timestamp as string.

Returns:
float: Duration in seconds between timestamps.
"""
# NTP timestamps as strings (representing seconds since 1900)
# convert the NTP timestamps from string to float
ntp_timestamp_1 = float(timestamp)
Expand Down
Loading
Loading