Saltar a contenido

Cron/Task Scheduler - SignalDashPro (Binance-first)#

Checklist para operar jobs automaticos de forma consistente en spot/futures.

Nota (deploy externo)#

En el VPS/Proxmox (CT110) estamos usando systemd timers en vez de cron para la operacion 30 dias:

  • signaldashpro-daily.timer: smoke + evidencia diaria (ver deploy/ops/daily.sh).
  • Ahora ejecuta el runner unificado scripts/run_spot_demo_daily.py (policies + DB health + checks ops + macro import opcional + research gate + evidencia + daily brief).
  • signaldashpro-alerts.timer: alertas operativas locales cada 5 min (ver deploy/ops/alerts.sh).
  • signaldashpro-storage-backup.timer: backup diario de storage/ (ver deploy/ops/storage_backup.sh).
  • signaldashpro-weekly-report.timer: genera reporte semanal (MD/JSON) los lunes 07:00 UTC (ver deploy/ops/weekly_report.sh).
  • signaldashpro-trade-smoke.timer: ejecuta un trade smoke controlado (Mon-Fri) en Spot Demo (ver deploy/ops/trade_smoke.sh).
  • signaldashpro-web-auth-smoke.timer: smoke autenticado (/auth/login + endpoints protegidos) cada 15 min (ver deploy/ops/web_auth_smoke.sh).
  • signaldashpro-capacity-watchdog.timer: watchdog de capacidad (disco/logs/docker/procesos críticos de backfill) cada hora (ver deploy/ops/capacity_watchdog.sh).
  • signaldashpro-orderbook-watchdog.timer: watchdog de ingesta order book cada 5 min (lag por símbolo, restart loop, presión de tablas; ver deploy/ops/orderbook_watchdog.sh).
  • signaldashpro-matvard-runtime-summary.timer: resumen MATVARD runtime cada 2h (orderbook health + daily auction brief + watchdog snapshot; ver deploy/ops/matvard_runtime_summary.sh).
  • signaldashpro-matvard-gate-snapshot.timer: snapshot MATVARD cada 2h (allow/skip/block, razones top, sesiones y símbolos; ver deploy/ops/matvard_gate_snapshot.sh).
  • signaldashpro-today-performance.timer: reporte operativo del día cada 2h (ver deploy/ops/today_performance.sh).

Comandos:

systemctl list-timers --all | grep signaldashpro
journalctl -u signaldashpro-daily.service -n 200 --no-pager
journalctl -u signaldashpro-capacity-watchdog.service -n 200 --no-pager
journalctl -u signaldashpro-orderbook-watchdog.service -n 200 --no-pager
journalctl -u signaldashpro-matvard-runtime-summary.service -n 200 --no-pager
journalctl -u signaldashpro-matvard-gate-snapshot.service -n 200 --no-pager

Watchdog de capacidad (CT110)#

Runner canónico:

/opt/signaldashpro/current/deploy/ops/capacity_watchdog.sh

Salida de reportes:

  • storage/logs/ops/ct110-capacity-watchdog-*.json
  • storage/logs/ops/ct110-capacity-watchdog-*.md
  • storage/logs/ops/ct110-capacity-watchdog-latest.{json,md}

Variables opcionales en .env:

  • SDP_WATCHDOG_DISK_WARN_PCT (default 75)
  • SDP_WATCHDOG_DISK_CRIT_PCT (default 85)
  • SDP_WATCHDOG_LOGS_WARN_GB (default 4)
  • SDP_WATCHDOG_LOGS_CRIT_GB (default 8)
  • SDP_WATCHDOG_DOCKER_RECLAIMABLE_WARN_GB (default 5)
  • SDP_WATCHDOG_AUTO_PRUNE_IF_CRIT (default false)
  • SDP_WATCHDOG_WARN_MISSING_PROCESS (default false)
  • SDP_WATCHDOG_REQUIRE_PROCESS_PATTERNS (CSV, opcional)

Watchdog de order book (CT110)#

Runner canónico:

/opt/signaldashpro/current/deploy/ops/orderbook_watchdog.sh

Salida de reportes:

  • storage/logs/ops/ct110-orderbook-watchdog-*.json
  • storage/logs/ops/ct110-orderbook-watchdog-*.md
  • storage/logs/ops/ct110-orderbook-watchdog-latest.{json,md}

Variables opcionales en .env:

  • SDP_ORDERBOOK_WATCHDOG_WINDOW_MINUTES (default 5)
  • SDP_ORDERBOOK_WATCHDOG_LAG_WARN_SEC (default 60)
  • SDP_ORDERBOOK_WATCHDOG_LAG_CRIT_SEC (default 180)
  • SDP_ORDERBOOK_WATCHDOG_MIN_DPM_WARN (default 0.5)
  • SDP_ORDERBOOK_WATCHDOG_MIN_DPM_CRIT (default 0.1)
  • SDP_ORDERBOOK_WATCHDOG_TABLE_WARN_MB (default 2048)
  • SDP_ORDERBOOK_WATCHDOG_TABLE_CRIT_MB (default 5120)
  • SDP_ORDERBOOK_WATCHDOG_CONTAINER (default deploy-microstructure-worker-1)

Resumen runtime MATVARD (CT110)#

Runner canónico:

/opt/signaldashpro/current/deploy/ops/matvard_runtime_summary.sh

Salida de reportes:

  • storage/logs/ops/matvard-runtime-summary-*.json
  • storage/logs/ops/matvard-runtime-summary-*.md
  • storage/logs/ops/matvard-runtime-summary-latest.{json,md}

Variables opcionales en .env:

  • MATVARD_SUMMARY_BRIEF_LOOKBACK_DAYS (default 7)
  • MATVARD_SUMMARY_BRIEF_LIMIT_PER_SYMBOL (default 5000)
  • MATVARD_SUMMARY_HEALTH_LOOKBACK_MINUTES (default 5)
  • MATVARD_SUMMARY_STALE_THRESHOLD_SECONDS (default 60)
  • MATVARD_SUMMARY_MIN_POINTS_1S (default 30)
  • MATVARD_SUMMARY_MIN_DPM (default 0.5)

Snapshot de gates MATVARD (CT110)#

Runner canónico:

/opt/signaldashpro/current/deploy/ops/matvard_gate_snapshot.sh

Salida de reportes:

  • storage/logs/ops/matvard-gate-snapshot-*.json
  • storage/logs/ops/matvard-gate-snapshot-*.md
  • storage/logs/ops/matvard-gate-snapshot-latest.{json,md}

Variables opcionales en .env:

  • MATVARD_GATE_SNAPSHOT_HOURS (default 2)
  • MATVARD_GATE_SNAPSHOT_STRATEGY_NAME (default autopilot:matvard_advanced_shadow)
  • MATVARD_GATE_SNAPSHOT_LIMIT (default 5000)

Jobs principales#

Job Variable *_AUTO_RUN Script/Comando Frecuencia sugerida Verificacion
Execution queue EXECUTION_QUEUE_AUTO_RUN Integrado en backend (SignalQueueService) EXECUTION_QUEUE_POLL_SECONDS GET /jobs/status + eventos processing_* en signal_queue_events.
News import NEWS_AUTO_RUN python scripts/import_market_news.py --symbols BTCUSDT ETHUSDT --brainstorm 15 min Logs en storage/logs/signaldashpro.log.
Macro import MACRO_AUTO_RUN python scripts/import_macro_events.py --days 7 --currencies EUR USD XAU 15 min Inserta eventos macro en macro_events segun MACRO_PROVIDER (tradingeconomics recomendado para demo).
News summary NEWS_SUMMARY_AUTO_RUN python scripts/import_market_news.py --summary-only --symbols BTCUSDT ETHUSDT 30 min Nuevos summary_ai en market_news.
Data pipeline DATA_PIPELINE_AUTO_RUN python scripts/run_data_pipeline.py --symbols BTCUSDT ETHUSDT --timeframe H1 60 min Nuevos registros en market_windows.
BI export BI_EXPORT_AUTO_RUN python scripts/export_bi_datasets.py --datasets trading_performance targets_progress --formats csv json --summary 4 h Archivos en storage/bi_exports/.
Training pipeline TRAINING_AUTO_RUN python scripts/run_training_pipeline.py --lookback-hours 240 --model-type random_forest Diario Nuevos artefactos en storage/models/.
Goal monitor META_GOAL_AUTO_RUN python scripts/run_goal_monitor.py --lookback-days 7 --alert-grace 3 60 min Nuevos goal_snapshots/goal_alerts.
External alerts (sin flag) python scripts/notify_external_alerts.py --backend-url http://127.0.0.1:8096 5 min Genera alertas en logs + estado en storage/alerts/alert_state.json.
Weekly report + ops audit (sin flag) python scripts/run_weekly_report.py + python scripts/run_ops_weekly_audit.py --backend-url http://127.0.0.1:8096 Semanal (lunes 07:00 UTC) Reporte semanal en storage/reports/weekly/ y auditoria en storage/logs/ops/weekly-audit-*.{json,md}.
Spot demo daily (sin flag) python scripts/run_spot_demo_daily.py --backend-url http://127.0.0.1:8096 --env-file env/.env.binance.demo.month --env-name binance-spot-demo Diario Ejecuta policy baseline + DB health + E2E + macro import opcional + research gate (con walk-forward por defecto) + evidencia + daily brief en un solo comando.
Ops daily brief (sin flag) python scripts/run_ops_daily_brief.py --backend-url http://127.0.0.1:8096 --env-name binance-spot-demo Diario Genera tablero breve (daily-brief-*.md/.json) en storage/logs/ops.
Spot trade smoke (sin flag) python scripts/run_spot_trade_smoke.py --backend-url http://127.0.0.1:8096 --symbol BTCUSDT --lot 0.001 --min-usdt 100 Diario (Mon-Fri) Crea/procesa 1 señal real controlada por dia en símbolo objetivo (sin legado EURUSDT).
Today performance (sin flag) bash deploy/ops/today_performance.sh Cada 2h Reporte rápido de allow/block/skip, cola, ejecuciones y live trades del día en storage/reports/ops/.
Web auth smoke (sin flag) python scripts/run_web_auth_smoke.py --frontend-url https://sdp.perlatec.net --api-url https://api.sdp.perlatec.net --email ... --password ... 15 min Valida login + endpoints protegidos y guarda snapshot en storage/logs/ops/web-auth-smoke-latest.json.

Variables recomendadas (Binance)#

  • EXECUTION_MODE=binance
  • EXECUTION_QUEUE_AUTO_RUN=true
  • EXECUTION_QUEUE_POLL_SECONDS=5
  • EXECUTION_QUEUE_BATCH=5
  • EXECUTION_RETRY_BACKOFF_BASE_SECONDS=5
  • EXECUTION_RETRY_BACKOFF_MAX_SECONDS=300
  • ALERT_BACKEND_BASE_URL=http://127.0.0.1:8096

Variables recomendadas (gates opportunity + auto-case)#

  • ALERT_GATES_OPPORTUNITY_ENABLED=true
  • ALERT_GATES_OPPORTUNITY_MIN_EVALUATED=20
  • ALERT_GATES_OPPORTUNITY_MAX_POTENTIAL_WIN_RATE_PCT=35
  • ALERT_GATES_OPPORTUNITY_AUTO_CASE_ENABLED=true
  • ALERT_GATES_OPPORTUNITY_AUTO_CASE_MIN_REPEATS=3
  • ALERT_GATES_OPPORTUNITY_AUTO_CASE_OWNER=ops_owner (opcional)
  • ALERT_GATES_OPPORTUNITY_AUTO_CASE_PRIORITY=high
  • ALERT_GATES_OPPORTUNITY_AUTO_CASE_DUE_HOURS=24

Parametros recomendados de gate diario#

Para scripts/run_spot_demo_daily.py:

  • --gate-profile production
  • --gate-symbols EURUSDT PAXGUSDT
  • --gate-timeframes H1
  • --gate-position-size 1
  • --gate-wf-enabled
  • --gate-wf-timeframes H1
  • --gate-wf-window-hours 168
  • --gate-wf-step-hours 24
  • --gate-wf-max-windows 4

Noticias: resiliencia y calidad#

  • Si FMP/NewsAPI/Prixe fallan, import_market_news.py aplica fallback a RSS (Google News) para no frenar el pipeline.
  • Variables para controlar ruido:
  • NEWS_MIN_RELEVANCE_SCORE (default 0.15)
  • NEWS_SOURCE_ALLOWLIST (opcional, coma separado)

Plantilla cron (Linux/macOS)#

SHELL=/bin/bash
BASH_ENV=/opt/signaldashpro/.env.binance
PATH=/usr/local/bin:/usr/bin:/bin
LOG_DIR=/var/log/signaldashpro

*/15 * * * * cd /opt/SignalDashPro && python scripts/import_market_news.py --symbols BTCUSDT ETHUSDT --brainstorm >> $LOG_DIR/cron.log 2>&1
*/30 * * * * cd /opt/SignalDashPro && python scripts/import_market_news.py --summary-only --symbols BTCUSDT ETHUSDT >> $LOG_DIR/cron.log 2>&1
0 * * * * cd /opt/SignalDashPro && python scripts/run_data_pipeline.py --symbols BTCUSDT ETHUSDT --timeframe H1 >> $LOG_DIR/cron.log 2>&1
0 */4 * * * cd /opt/SignalDashPro && python scripts/export_bi_datasets.py --datasets trading_performance targets_progress --formats csv json --summary >> $LOG_DIR/cron.log 2>&1
30 2 * * * cd /opt/SignalDashPro && python scripts/run_training_pipeline.py --lookback-hours 240 --model-type random_forest >> $LOG_DIR/cron.log 2>&1
*/60 * * * * cd /opt/SignalDashPro && python scripts/run_goal_monitor.py --lookback-days 7 --alert-grace 3 >> $LOG_DIR/cron.log 2>&1
*/5 * * * * cd /opt/SignalDashPro && python scripts/notify_external_alerts.py --backend-url http://127.0.0.1:8096 >> $LOG_DIR/cron.log 2>&1
0 7 * * 1 cd /opt/SignalDashPro && python scripts/run_weekly_report.py >> $LOG_DIR/cron.log 2>&1

Plantilla Task Scheduler (Windows)#

Ejecutable:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Arguments (ejemplo pipeline):

-NoProfile -ExecutionPolicy Bypass -Command "Set-Location 'D:\Repos\SignalDashPro'; . .\setenv.ps1 env\.env.binance; python scripts\run_data_pipeline.py --symbols BTCUSDT ETHUSDT --timeframe H1 >> .\storage\logs\cron.log 2>&1"

Evidencia operativa#

Captura snapshot operativo (HTTP + DB) y guarda JSON en storage/logs/evidence/:

set -a; source env/.env.binance; set +a
.venv/bin/python scripts/capture_operational_evidence.py --env-name binance-spot --backend-url http://127.0.0.1:8096

Campos clave a revisar en la evidencia:

  • /jobs/status (execution_queue.auto_run=true)
  • /risk/kill-switch/status (enabled=false en operacion normal)
  • /binance/status (market/symbols/limits esperados)
  • db_health.checks sin warnings criticos

Visualizacion recomendada en frontend:

  • /control-tower: estado en vivo de robot/gate/jobs/queue/news/brief.
  • /ops: evidencia y logs operativos 24/7.