I found out a person from a developer portal company I was looking into had recently moved to OpenObserve. I was chatting with them on LinkedIn about other topics but figured I should really give OpenObserve a look as I’ve often looked at Observability tools, small and large, over the years.
Company
OpenObserve is a relative up and comer started in 2022 by Prabhat Sharma as ZincSearch before pivoting to Observability. Its still relatively small with an estimated 11-50 employees and based in Menlo Park, CA. That said, it’s recently closed some Series A in April with Nexus Venture Partners and Dell Tech Capital so perhaps it’s poised to grow.
Install
We can start by looking at Docker
It’s easy to launch the latest looking at the install steps.
A slight nuance to their steps is that we need to first pull the image from a public ECR
$ docker pull public.ecr.aws/zinclabs/openobserve:v0.91.1
v0.91.1: Pulling from zinclabs/openobserve
1c193acf1cd1: Pull complete
5b3013706a46: Pull complete
8029962f461f: Pull complete
9cd8e28f1c22: Pull complete
9d2753d762cc: Pull complete
0f8b424aa0b9: Pull complete
d557676654e5: Pull complete
d82bc7a76a83: Pull complete
68e4cd60e60f: Pull complete
0783a66ae749: Pull complete
efae86a3fb38: Pull complete
411e1c946a02: Pull complete
77964957095d: Pull complete
71f0f831b11e: Pull complete
ab5b9f04e505: Pull complete
6c174cbb5f6c: Pull complete
b62c759db193: Pull complete
80a6d6970a05: Pull complete
68f04394073c: Pull complete
78269c9827a3: Pull complete
29e9d2040d07: Pull complete
87eca5491eb5: Pull complete
807b853d83ce: Pull complete
6d096c718601: Pull complete
f74bf462ff02: Pull complete
3a537e5acd26: Pull complete
Digest: sha256:e1ff0445fab3e748ac4cf630308cc8493579e50d19ad255bb3a3b8c1b710aaf7
Status: Downloaded newer image for public.ecr.aws/zinclabs/openobserve:v0.91.1
public.ecr.aws/zinclabs/openobserve:v0.91.1
Then we can launch it
$ docker run -v $PWD/data:/data -e ZO_DATA_DIR="/data" -p 5080:5080 -e ZO_ROOT_USER_EMAIL="root@example.com" -e ZO_ROOT_USER_PASSWORD="Complexpass#123" public.ecr.aws/zinclabs/openobserve:v0.91.1
2026-07-07T20:21:35.529039610+00:00 INFO openobserve: Starting OpenObserve v0.91.1
2026-07-07T20:21:35.529139250+00:00 INFO openobserve: System info: CPU cores 16, MEM total 31.31 GB, Disk total 250.92 GB, free 8.55 GB
2026-07-07T20:21:35.529145893+00:00 INFO openobserve: Caches info: Disk max size 4.27 GB, MEM max size 7.83 GB, Datafusion pool size: 11.74 GB
2026-07-07T20:21:35.533878652+00:00 INFO config::ider: init ider with machine_id: 0
2026-07-07T20:21:35.535092618+00:00 WARN openobserve::migration: error in getting db schema version DbError# error error returned from database: (code: 14) unable to open database file performing operation on key /db_schema_version/; assuming default of 0 and trying upgrade.
2026-07-07T20:21:35.535132855+00:00 INFO openobserve::migration: DB_SCHEMA_VERSION mismatch : expected 45, found 0; running db upgrade
2026-07-07T20:21:35.547443495+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_start_dt_idx on table meta
2026-07-07T20:21:35.547823372+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_start_dt_idx created successfully
2026-07-07T20:21:35.547849252+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_idx on table meta
2026-07-07T20:21:35.548279627+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_idx created successfully
2026-07-07T20:21:35.548304194+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_key1_idx on table meta
2026-07-07T20:21:35.548716354+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_key1_idx created successfully
2026-07-07T20:21:35.548751010+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_start_dt_idx on table meta
2026-07-07T20:21:35.549031964+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_start_dt_idx created successfully
2026-07-07T20:21:35.549476410+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_start_dt_idx on table meta
2026-07-07T20:21:35.549715739+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_start_dt_idx created successfully
2026-07-07T20:21:35.549741899+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_idx on table meta
2026-07-07T20:21:35.549960798+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_idx created successfully
2026-07-07T20:21:35.549989914+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_key1_idx on table meta
2026-07-07T20:21:35.550202742+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_key1_idx created successfully
2026-07-07T20:21:35.550238140+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_start_dt_idx on table meta
2026-07-07T20:21:35.550459945+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_start_dt_idx created successfully
2026-07-07T20:21:35.556978752+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_org_idx on table file_list
2026-07-07T20:21:35.557326332+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_org_idx created successfully
2026-07-07T20:21:35.557351480+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_stream_ts_idx on table file_list
2026-07-07T20:21:35.557805230+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_stream_ts_idx created successfully
2026-07-07T20:21:35.557832562+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_stream_date_idx on table file_list
2026-07-07T20:21:35.558283536+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_stream_date_idx created successfully
2026-07-07T20:21:35.558310418+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_updated_at_deleted_idx on table file_list
2026-07-07T20:21:35.558783564+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_updated_at_deleted_idx created successfully
2026-07-07T20:21:35.558807941+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_history_org_idx on table file_list_history
2026-07-07T20:21:35.559219289+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_history_org_idx created successfully
2026-07-07T20:21:35.559248134+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_history_stream_ts_idx on table file_list_history
2026-07-07T20:21:35.559676344+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_history_stream_ts_idx created successfully
2026-07-07T20:21:35.559701663+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_deleted_created_at_idx on table file_list_deleted
2026-07-07T20:21:35.560251327+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_deleted_created_at_idx created successfully
2026-07-07T20:21:35.560275333+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_deleted_stream_date_file_idx on table file_list_deleted
2026-07-07T20:21:35.560671742+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_deleted_stream_date_file_idx created successfully
2026-07-07T20:21:35.560697030+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_jobs_stream_status_idx on table file_list_jobs
2026-07-07T20:21:35.561113809+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_jobs_stream_status_idx created successfully
2026-07-07T20:21:35.561143256+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_jobs_status_dumped_idx on table file_list_jobs
2026-07-07T20:21:35.561530348+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_jobs_status_dumped_idx created successfully
2026-07-07T20:21:35.561555396+00:00 INFO infra::db::sqlite: [SQLITE] creating index stream_stats_org_idx on table stream_stats
2026-07-07T20:21:35.561958258+00:00 INFO infra::db::sqlite: [SQLITE] index stream_stats_org_idx created successfully
2026-07-07T20:21:35.561981412+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_dump_stats_org_idx on table file_list_dump_stats
2026-07-07T20:21:35.562435618+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_dump_stats_org_idx created successfully
2026-07-07T20:21:35.562460727+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_history_stream_file_idx on table file_list_history
2026-07-07T20:21:35.562953695+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_history_stream_file_idx created successfully
2026-07-07T20:21:35.562978913+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_jobs_stream_offsets_idx on table file_list_jobs
2026-07-07T20:21:35.563482729+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_jobs_stream_offsets_idx created successfully
2026-07-07T20:21:35.563508749+00:00 INFO infra::db::sqlite: [SQLITE] creating index stream_stats_org_stream_recent_idx on table stream_stats
2026-07-07T20:21:35.563922240+00:00 INFO infra::db::sqlite: [SQLITE] index stream_stats_org_stream_recent_idx created successfully
2026-07-07T20:21:35.563952528+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_dump_stats_stream_file_idx on table file_list_dump_stats
2026-07-07T20:21:35.564364078+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_dump_stats_stream_file_idx created successfully
2026-07-07T20:21:35.564387984+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_stream_file_idx on table file_list
2026-07-07T20:21:35.564873764+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_stream_file_idx created successfully
2026-07-07T20:21:35.567713005+00:00 INFO infra::db::sqlite: [SQLITE] creating index scheduled_jobs_key_idx on table scheduled_jobs
2026-07-07T20:21:35.568021637+00:00 INFO infra::db::sqlite: [SQLITE] index scheduled_jobs_key_idx created successfully
2026-07-07T20:21:35.568048097+00:00 INFO infra::db::sqlite: [SQLITE] creating index scheduled_jobs_org_key_idx on table scheduled_jobs
2026-07-07T20:21:35.568490905+00:00 INFO infra::db::sqlite: [SQLITE] index scheduled_jobs_org_key_idx created successfully
2026-07-07T20:21:35.568517867+00:00 INFO infra::db::sqlite: [SQLITE] creating index scheduled_jobs_org_module_key_idx on table scheduled_jobs
2026-07-07T20:21:35.569076928+00:00 INFO infra::db::sqlite: [SQLITE] index scheduled_jobs_org_module_key_idx created successfully
2026-07-07T20:21:35.569624457+00:00 INFO infra::db::sqlite: [SQLITE] creating index schema_history_org_idx on table schema_history
2026-07-07T20:21:35.570046646+00:00 INFO infra::db::sqlite: [SQLITE] index schema_history_org_idx created successfully
2026-07-07T20:21:35.570072195+00:00 INFO infra::db::sqlite: [SQLITE] creating index schema_history_stream_idx on table schema_history
2026-07-07T20:21:35.570539872+00:00 INFO infra::db::sqlite: [SQLITE] index schema_history_stream_idx created successfully
2026-07-07T20:21:35.570573076+00:00 INFO infra::db::sqlite: [SQLITE] creating index schema_history_stream_version_idx on table schema_history
2026-07-07T20:21:35.571028368+00:00 INFO infra::db::sqlite: [SQLITE] index schema_history_stream_version_idx created successfully
2026-07-07T20:21:35.572071927+00:00 INFO infra::db::sqlite: [SQLITE] creating index short_urls_short_id_idx on table short_urls
2026-07-07T20:21:35.572591463+00:00 INFO infra::db::sqlite: [SQLITE] index short_urls_short_id_idx created successfully
2026-07-07T20:21:35.572618545+00:00 INFO infra::db::sqlite: [SQLITE] creating index short_urls_created_ts_idx on table short_urls
2026-07-07T20:21:35.573052375+00:00 INFO infra::db::sqlite: [SQLITE] index short_urls_created_ts_idx created successfully
2026-07-07T20:21:35.574805295+00:00 INFO sea_orm_migration::migrator: Applying 12 pending migrations
2026-07-07T20:21:35.575799440+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241114_000001_create_folders_table'
2026-07-07T20:21:35.577238026+00:00 INFO sea_orm_migration::migrator: Migration 'm20241114_000001_create_folders_table' has been applied
2026-07-07T20:21:35.577767351+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241115_150000_populate_folders_table'
2026-07-07T20:21:35.578319148+00:00 INFO sea_orm_migration::migrator: Migration 'm20241115_150000_populate_folders_table' has been applied
2026-07-07T20:21:35.578627118+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241116_000001_delete_metas'
2026-07-07T20:21:35.578923917+00:00 INFO sea_orm_migration::migrator: Migration 'm20241116_000001_delete_metas' has been applied
2026-07-07T20:21:35.579189536+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241116_000002_drop_folders_created_at_column'
2026-07-07T20:21:35.580423220+00:00 INFO sea_orm_migration::migrator: Migration 'm20241116_000002_drop_folders_created_at_column' has been applied
2026-07-07T20:21:35.580871839+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241119_000001_create_dashboards_table'
2026-07-07T20:21:35.581723377+00:00 INFO sea_orm_migration::migrator: Migration 'm20241119_000001_create_dashboards_table' has been applied
2026-07-07T20:21:35.582234661+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241119_000002_populate_dashboards_table'
2026-07-07T20:21:35.582824762+00:00 INFO sea_orm_migration::migrator: Migration 'm20241119_000002_populate_dashboards_table' has been applied
2026-07-07T20:21:35.583052638+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241119_000003_delete_metas'
2026-07-07T20:21:35.583502520+00:00 INFO sea_orm_migration::migrator: Migration 'm20241119_000003_delete_metas' has been applied
2026-07-07T20:21:35.583769081+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241204_143100_create_table_search_queue'
2026-07-07T20:21:35.585096006+00:00 INFO sea_orm_migration::migrator: Migration 'm20241204_143100_create_table_search_queue' has been applied
2026-07-07T20:21:35.585567931+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241209_120000_create_alerts_table'
2026-07-07T20:21:35.587129323+00:00 INFO sea_orm_migration::migrator: Migration 'm20241209_120000_create_alerts_table' has been applied
2026-07-07T20:21:35.587632464+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241215_190333_delete_metas'
2026-07-07T20:21:35.588053511+00:00 INFO sea_orm_migration::migrator: Migration 'm20241215_190333_delete_metas' has been applied
2026-07-07T20:21:35.588418259+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241217_154900_alter_folders_table_idx'
2026-07-07T20:21:35.589431794+00:00 INFO sea_orm_migration::migrator: Migration 'm20241217_154900_alter_folders_table_idx' has been applied
2026-07-07T20:21:35.589947440+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241217_155000_populate_alerts_table'
2026-07-07T20:21:35.590722638+00:00 INFO sea_orm_migration::migrator: Migration 'm20241217_155000_populate_alerts_table' has been applied
2026-07-07T20:21:35.591206433+00:00 INFO sea_orm_migration::migrator: Applying all pending migrations
2026-07-07T20:21:35.592208776+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241222_085111_search_jobs'
2026-07-07T20:21:35.592636593+00:00 INFO sea_orm_migration::migrator: Migration 'm20241222_085111_search_jobs' has been applied
2026-07-07T20:21:35.593226977+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241222_085135_search_job_partitions'
2026-07-07T20:21:35.593609196+00:00 INFO sea_orm_migration::migrator: Migration 'm20241222_085135_search_job_partitions' has been applied
2026-07-07T20:21:35.594210179+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241222_085148_search_job_results'
2026-07-07T20:21:35.594571116+00:00 INFO sea_orm_migration::migrator: Migration 'm20241222_085148_search_job_results' has been applied
2026-07-07T20:21:35.595115674+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241227_000001_create_organizations_table'
2026-07-07T20:21:35.595606995+00:00 INFO sea_orm_migration::migrator: Migration 'm20241227_000001_create_organizations_table' has been applied
2026-07-07T20:21:35.596165796+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241227_000100_populate_organizations_table'
2026-07-07T20:21:35.596700040+00:00 INFO sea_orm_migration::migrator: Migration 'm20241227_000100_populate_organizations_table' has been applied
2026-07-07T20:21:35.596957883+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241227_000200_create_users_table'
2026-07-07T20:21:35.598048075+00:00 INFO sea_orm_migration::migrator: Migration 'm20241227_000200_create_users_table' has been applied
2026-07-07T20:21:35.598705194+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241227_000300_create_org_users_table'
2026-07-07T20:21:35.600120797+00:00 INFO sea_orm_migration::migrator: Migration 'm20241227_000300_create_org_users_table' has been applied
2026-07-07T20:21:35.600661112+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20241227_000400_populate_users_table'
2026-07-07T20:21:35.601214480+00:00 INFO sea_orm_migration::migrator: Migration 'm20241227_000400_populate_users_table' has been applied
2026-07-07T20:21:35.601434081+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250107_160900_delete_bad_dashboards'
2026-07-07T20:21:35.601457165+00:00 INFO sea_orm_migration::migrator: Migration 'm20250107_160900_delete_bad_dashboards' has been applied
2026-07-07T20:21:35.601706682+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250109_092400_recreate_tables_with_ksuids'
2026-07-07T20:21:35.615985718+00:00 INFO sea_orm_migration::migrator: Migration 'm20250109_092400_recreate_tables_with_ksuids' has been applied
2026-07-07T20:21:35.616623347+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250113_144600_create_unique_folder_name_idx'
2026-07-07T20:21:35.617440824+00:00 INFO sea_orm_migration::migrator: Migration 'm20250113_144600_create_unique_folder_name_idx' has been applied
2026-07-07T20:21:35.618058938+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250121_120000_create_cipher_table'
2026-07-07T20:21:35.618401865+00:00 INFO sea_orm_migration::migrator: Migration 'm20250121_120000_create_cipher_table' has been applied
2026-07-07T20:21:35.619137724+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250122_000001_create_table_action_scripts'
2026-07-07T20:21:35.619529575+00:00 INFO sea_orm_migration::migrator: Migration 'm20250122_000001_create_table_action_scripts' has been applied
2026-07-07T20:21:35.620266478+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250124_000001_create_timed_annotations_table'
2026-07-07T20:21:35.621325910+00:00 INFO sea_orm_migration::migrator: Migration 'm20250124_000001_create_timed_annotations_table' has been applied
2026-07-07T20:21:35.621957508+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250124_000002_create_timed_annotation_panels_table'
2026-07-07T20:21:35.623657686+00:00 INFO sea_orm_migration::migrator: Migration 'm20250124_000002_create_timed_annotation_panels_table' has been applied
2026-07-07T20:21:35.624264128+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250125_115400_create_templates_table'
2026-07-07T20:21:35.625290237+00:00 INFO sea_orm_migration::migrator: Migration 'm20250125_115400_create_templates_table' has been applied
2026-07-07T20:21:35.625898640+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250125_132500_populate_templates_table'
2026-07-07T20:21:35.626423095+00:00 INFO sea_orm_migration::migrator: Migration 'm20250125_132500_populate_templates_table' has been applied
2026-07-07T20:21:35.626729913+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250125_172300_delete_metas_templates'
2026-07-07T20:21:35.627196938+00:00 INFO sea_orm_migration::migrator: Migration 'm20250125_172300_delete_metas_templates' has been applied
2026-07-07T20:21:35.627460833+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250125_102300_create_destinations_table'
2026-07-07T20:21:35.628518884+00:00 INFO sea_orm_migration::migrator: Migration 'm20250125_102300_create_destinations_table' has been applied
2026-07-07T20:21:35.629179059+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250125_133700_populate_destinations_table'
2026-07-07T20:21:35.629756505+00:00 INFO sea_orm_migration::migrator: Migration 'm20250125_133700_populate_destinations_table' has been applied
2026-07-07T20:21:35.629999328+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250125_153005_delete_metas_destinations'
2026-07-07T20:21:35.630467806+00:00 INFO sea_orm_migration::migrator: Migration 'm20250125_153005_delete_metas_destinations' has been applied
2026-07-07T20:21:35.630743895+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250213_000001_add_dashboard_updated_at'
2026-07-07T20:21:35.631375517+00:00 INFO sea_orm_migration::migrator: Migration 'm20250213_000001_add_dashboard_updated_at' has been applied
2026-07-07T20:21:35.632019712+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250217_115548_ratelimit_table'
2026-07-07T20:21:35.633219330+00:00 INFO sea_orm_migration::migrator: Migration 'm20250217_115548_ratelimit_table' has been applied
2026-07-07T20:21:35.633837002+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250320_000001_remove_alert_name_unique_constraint'
2026-07-07T20:21:35.635097367+00:00 INFO sea_orm_migration::migrator: Migration 'm20250320_000001_remove_alert_name_unique_constraint' has been applied
2026-07-07T20:21:35.635386+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250422_000001_add_alert_align_time'
2026-07-07T20:21:35.636132603+00:00 INFO sea_orm_migration::migrator: Migration 'm20250422_000001_add_alert_align_time' has been applied
2026-07-07T20:21:35.636757017+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250520_000001_add_trial_end_col'
2026-07-07T20:21:35.636779149+00:00 INFO sea_orm_migration::migrator: Migration 'm20250520_000001_add_trial_end_col' has been applied
2026-07-07T20:21:35.637018881+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250611_000001_create_reports_table'
2026-07-07T20:21:35.638902669+00:00 INFO sea_orm_migration::migrator: Migration 'm20250611_000001_create_reports_table' has been applied
2026-07-07T20:21:35.639536092+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250611_000002_populate_reports_table'
2026-07-07T20:21:35.640177531+00:00 INFO sea_orm_migration::migrator: Migration 'm20250611_000002_populate_reports_table' has been applied
2026-07-07T20:21:35.640400078+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250611_000003_populate_reports_scheduled_jobs'
2026-07-07T20:21:35.640670946+00:00 INFO sea_orm_migration::migrator: Migration 'm20250611_000003_populate_reports_scheduled_jobs' has been applied
2026-07-07T20:21:35.640933269+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250612_000001_create_re_pattern_table'
2026-07-07T20:21:35.642307803+00:00 INFO sea_orm_migration::migrator: Migration 'm20250612_000001_create_re_pattern_table' has been applied
2026-07-07T20:21:35.643146352+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250612_000002_create_re_pattern_stream_map_table'
2026-07-07T20:21:35.643499367+00:00 INFO sea_orm_migration::migrator: Migration 'm20250612_000002_create_re_pattern_stream_map_table' has been applied
2026-07-07T20:21:35.644339947+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250716_000001_create_enrichment_table'
2026-07-07T20:21:35.646140319+00:00 INFO sea_orm_migration::migrator: Migration 'm20250716_000001_create_enrichment_table' has been applied
2026-07-07T20:21:35.646854448+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250731_000001_create_compactor_manual_jobs'
2026-07-07T20:21:35.647919238+00:00 INFO sea_orm_migration::migrator: Migration 'm20250731_000001_create_compactor_manual_jobs' has been applied
2026-07-07T20:21:35.648593089+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250801_000001_create_system_prompts_table'
2026-07-07T20:21:35.649021608+00:00 INFO sea_orm_migration::migrator: Migration 'm20250801_000001_create_system_prompts_table' has been applied
2026-07-07T20:21:35.649708427+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250822_093713_add_updated_at_for_file_list_table'
2026-07-07T20:21:35.650408045+00:00 INFO sea_orm_migration::migrator: Migration 'm20250822_093713_add_updated_at_for_file_list_table' has been applied
2026-07-07T20:21:35.650699964+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250923_000001_update_enrichment_table_data_size'
2026-07-07T20:21:35.650724131+00:00 INFO sea_orm_migration::migrator: Migration 'm20250923_000001_update_enrichment_table_data_size' has been applied
2026-07-07T20:21:35.650983450+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20250930_000001_create_pipeline_last_errors_table'
2026-07-07T20:21:35.652099787+00:00 INFO sea_orm_migration::migrator: Migration 'm20250930_000001_create_pipeline_last_errors_table' has been applied
2026-07-07T20:21:35.652838985+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251024_000001_add_alert_deduplication'
2026-07-07T20:21:35.660112310+00:00 INFO sea_orm_migration::migrator: Migration 'm20251024_000001_add_alert_deduplication' has been applied
2026-07-07T20:21:35.660828062+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251105_000001_update_enrichment_table_created_at_mysql'
2026-07-07T20:21:35.660852849+00:00 INFO sea_orm_migration::migrator: Migration 'm20251105_000001_update_enrichment_table_created_at_mysql' has been applied
2026-07-07T20:21:35.661107718+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251118_000001_add_alert_row_template_type'
2026-07-07T20:21:35.661851285+00:00 INFO sea_orm_migration::migrator: Migration 'm20251118_000001_add_alert_row_template_type' has been applied
2026-07-07T20:21:35.662718566+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251118_000002_create_sessions_table'
2026-07-07T20:21:35.663077312+00:00 INFO sea_orm_migration::migrator: Migration 'm20251118_000002_create_sessions_table' has been applied
2026-07-07T20:21:35.663860372+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251118_000003_delete_meta_sessions'
2026-07-07T20:21:35.664303050+00:00 INFO sea_orm_migration::migrator: Migration 'm20251118_000003_delete_meta_sessions' has been applied
2026-07-07T20:21:35.664550044+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251126_100001_create_service_streams_table'
2026-07-07T20:21:35.666367907+00:00 INFO sea_orm_migration::migrator: Migration 'm20251126_100001_create_service_streams_table' has been applied
2026-07-07T20:21:35.667264014+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251126_100002_create_service_streams_dimensions_table'
2026-07-07T20:21:35.668469937+00:00 INFO sea_orm_migration::migrator: Migration 'm20251126_100002_create_service_streams_dimensions_table' has been applied
2026-07-07T20:21:35.669215696+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251128_000001_create_kv_store_table'
2026-07-07T20:21:35.669577146+00:00 INFO sea_orm_migration::migrator: Migration 'm20251128_000001_create_kv_store_table' has been applied
2026-07-07T20:21:35.670340492+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251128_000002_populate_kv_store_table'
2026-07-07T20:21:35.670959726+00:00 INFO sea_orm_migration::migrator: Migration 'm20251128_000002_populate_kv_store_table' has been applied
2026-07-07T20:21:35.671204647+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251128_000003_delete_kv_from_meta'
2026-07-07T20:21:35.671624740+00:00 INFO sea_orm_migration::migrator: Migration 'm20251128_000003_delete_kv_from_meta' has been applied
2026-07-07T20:21:35.671868597+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251204_000001_create_alert_incidents_table'
2026-07-07T20:21:35.677861874+00:00 INFO sea_orm_migration::migrator: Migration 'm20251204_000001_create_alert_incidents_table' has been applied
2026-07-07T20:21:35.678788129+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251207_000001_create_system_settings_table'
2026-07-07T20:21:35.684092844+00:00 INFO sea_orm_migration::migrator: Migration 'm20251207_000001_create_system_settings_table' has been applied
2026-07-07T20:21:35.684956378+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251219_000001_add_org_id_to_search_queue'
2026-07-07T20:21:35.686680963+00:00 INFO sea_orm_migration::migrator: Migration 'm20251219_000001_add_org_id_to_search_queue' has been applied
2026-07-07T20:21:35.687520401+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251221_000001_create_enrichment_table_urls'
2026-07-07T20:21:35.688792667+00:00 INFO sea_orm_migration::migrator: Migration 'm20251221_000001_create_enrichment_table_urls' has been applied
2026-07-07T20:21:35.689612220+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251218_000001_add_expires_at_to_sessions'
2026-07-07T20:21:35.691385255+00:00 INFO sea_orm_migration::migrator: Migration 'm20251218_000001_add_expires_at_to_sessions' has been applied
2026-07-07T20:21:35.692366595+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251226_000001_add_enrichment_table_urls_is_local_region'
2026-07-07T20:21:35.693315556+00:00 INFO sea_orm_migration::migrator: Migration 'm20251226_000001_add_enrichment_table_urls_is_local_region' has been applied
2026-07-07T20:21:35.694209898+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251229_000001_create_backfill_jobs'
2026-07-07T20:21:35.696414244+00:00 INFO sea_orm_migration::migrator: Migration 'm20251229_000001_create_backfill_jobs' has been applied
2026-07-07T20:21:35.697187004+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20251230_000001_add_allow_static_token_to_org_users'
2026-07-07T20:21:35.698157602+00:00 INFO sea_orm_migration::migrator: Migration 'm20251230_000001_add_allow_static_token_to_org_users' has been applied
2026-07-07T20:21:35.699108614+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260107_000001_sync_distinct_stream_retention'
2026-07-07T20:21:35.700137726+00:00 INFO sea_orm_migration::migrator: Migration 'm20260107_000001_sync_distinct_stream_retention' has been applied
2026-07-07T20:21:35.700465474+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260108_000001_recreate_enrichment_table_urls_with_ksuids'
2026-07-07T20:21:35.710869653+00:00 INFO sea_orm_migration::migrator: Migration 'm20260108_000001_recreate_enrichment_table_urls_with_ksuids' has been applied
2026-07-07T20:21:35.712014985+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260113_000001_add_alert_template'
2026-07-07T20:21:35.712988761+00:00 INFO sea_orm_migration::migrator: Migration 'm20260113_000001_add_alert_template' has been applied
2026-07-07T20:21:35.713860083+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260116_000001_add_enabled_to_backfill_jobs'
2026-07-07T20:21:35.714823367+00:00 INFO sea_orm_migration::migrator: Migration 'm20260116_000001_add_enabled_to_backfill_jobs' has been applied
2026-07-07T20:21:35.715711451+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260119_000001_add_stat_interval_to_ratelimit'
2026-07-07T20:21:35.718695409+00:00 INFO sea_orm_migration::migrator: Migration 'm20260119_000001_add_stat_interval_to_ratelimit' has been applied
2026-07-07T20:21:35.719520470+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260121_00001_create_sourcemap_table'
2026-07-07T20:21:35.720978531+00:00 INFO sea_orm_migration::migrator: Migration 'm20260121_00001_create_sourcemap_table' has been applied
2026-07-07T20:21:35.721862698+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260131_000001_add_unique_constraint_templates_org_name'
2026-07-07T20:21:35.722194554+00:00 INFO sea_orm_migration::migrator: Migration 'm20260131_000001_add_unique_constraint_templates_org_name' has been applied
2026-07-07T20:21:35.723111679+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260212_000001_widen_incident_correlation_key'
2026-07-07T20:21:35.723135074+00:00 INFO sea_orm_migration::migrator: Migration 'm20260212_000001_widen_incident_correlation_key' has been applied
2026-07-07T20:21:35.723407375+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260214_000001_create_incident_events_table'
2026-07-07T20:21:35.723770704+00:00 INFO sea_orm_migration::migrator: Migration 'm20260214_000001_create_incident_events_table' has been applied
2026-07-07T20:21:35.724797579+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260227_000001_add_alert_creates_incident'
2026-07-07T20:21:35.725966652+00:00 INFO sea_orm_migration::migrator: Migration 'm20260227_000001_add_alert_creates_incident' has been applied
2026-07-07T20:21:35.727152875+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260305_000001_create_trial_quota_usage_table'
2026-07-07T20:21:35.727536157+00:00 INFO sea_orm_migration::migrator: Migration 'm20260305_000001_create_trial_quota_usage_table' has been applied
2026-07-07T20:21:35.728552004+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260310_000001_create_anomaly_detection_config_table'
2026-07-07T20:21:35.731841578+00:00 INFO sea_orm_migration::migrator: Migration 'm20260310_000001_create_anomaly_detection_config_table' has been applied
2026-07-07T20:21:35.732752703+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260310_000002_create_anomaly_detection_models_table'
2026-07-07T20:21:35.734248019+00:00 INFO sea_orm_migration::migrator: Migration 'm20260310_000002_create_anomaly_detection_models_table' has been applied
2026-07-07T20:21:35.735505749+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260312_000001_create_eval_templates_table'
2026-07-07T20:21:35.740165751+00:00 INFO sea_orm_migration::migrator: Migration 'm20260312_000001_create_eval_templates_table' has been applied
2026-07-07T20:21:35.741056828+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260317_000001_add_anomaly_detection_config_columns'
2026-07-07T20:21:35.752006159+00:00 INFO sea_orm_migration::migrator: Migration 'm20260317_000001_add_anomaly_detection_config_columns' has been applied
2026-07-07T20:21:35.752914097+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260318_000001_recreate_service_streams_schema'
2026-07-07T20:21:35.756264138+00:00 INFO sea_orm_migration::migrator: Migration 'm20260318_000001_recreate_service_streams_schema' has been applied
2026-07-07T20:21:35.757250378+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260318_000002_drop_service_streams_dimensions'
2026-07-07T20:21:35.757606631+00:00 INFO sea_orm_migration::migrator: Migration 'm20260318_000002_drop_service_streams_dimensions' has been applied
2026-07-07T20:21:35.758722747+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260318_000003_alter_alert_incidents_schema'
2026-07-07T20:21:35.769758985+00:00 INFO sea_orm_migration::migrator: Migration 'm20260318_000003_alter_alert_incidents_schema' has been applied
2026-07-07T20:21:35.770713055+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260318_000004_add_set_id_to_service_streams'
2026-07-07T20:21:35.771689807+00:00 INFO sea_orm_migration::migrator: Migration 'm20260318_000004_add_set_id_to_service_streams' has been applied
2026-07-07T20:21:35.772554033+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260318_000005_add_all_dimensions_to_service_streams'
2026-07-07T20:21:35.773445952+00:00 INFO sea_orm_migration::migrator: Migration 'm20260318_000005_add_all_dimensions_to_service_streams' has been applied
2026-07-07T20:21:35.774304767+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260326_000001_add_field_name_mapping_to_service_streams'
2026-07-07T20:21:35.775349910+00:00 INFO sea_orm_migration::migrator: Migration 'm20260326_000001_add_field_name_mapping_to_service_streams' has been applied
2026-07-07T20:21:35.776338334+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260331_000001_create_sys_rca_agent_service_accounts'
2026-07-07T20:21:35.776851104+00:00 INFO sea_orm_migration::migrator: Migration 'm20260331_000001_create_sys_rca_agent_service_accounts' has been applied
2026-07-07T20:21:35.777162080+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260402_000001_reports_add_image_fields'
2026-07-07T20:21:35.782911798+00:00 INFO sea_orm_migration::migrator: Migration 'm20260402_000001_reports_add_image_fields' has been applied
2026-07-07T20:21:35.783899893+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260414_000001_add_is_system_to_cipher_keys'
2026-07-07T20:21:35.784851156+00:00 INFO sea_orm_migration::migrator: Migration 'm20260414_000001_add_is_system_to_cipher_keys' has been applied
2026-07-07T20:21:35.785792610+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260414_000002_create_org_ai_toolsets'
2026-07-07T20:21:35.787178877+00:00 INFO sea_orm_migration::migrator: Migration 'm20260414_000002_create_org_ai_toolsets' has been applied
2026-07-07T20:21:35.788041239+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260415_000001_create_model_pricing_table'
2026-07-07T20:21:35.789452223+00:00 INFO sea_orm_migration::migrator: Migration 'm20260415_000001_create_model_pricing_table' has been applied
2026-07-07T20:21:35.790382746+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260415_000002_add_source_to_model_pricing'
2026-07-07T20:21:35.795675267+00:00 INFO sea_orm_migration::migrator: Migration 'm20260415_000002_add_source_to_model_pricing' has been applied
2026-07-07T20:21:35.795988204+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260504_000001_add_anomaly_detection_config_folder_fk'
2026-07-07T20:21:35.796014615+00:00 INFO sea_orm_migration::migrator: Migration 'm20260504_000001_add_anomaly_detection_config_folder_fk' has been applied
2026-07-07T20:21:35.796333696+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260506_000001_create_org_ingestion_tokens_table'
2026-07-07T20:21:35.800090918+00:00 INFO sea_orm_migration::migrator: Migration 'm20260506_000001_create_org_ingestion_tokens_table' has been applied
2026-07-07T20:21:35.801228598+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260401_00001_create_org_s3_table'
2026-07-07T20:21:35.801601260+00:00 INFO sea_orm_migration::migrator: Migration 'm20260401_00001_create_org_s3_table' has been applied
2026-07-07T20:21:35.802616716+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260520_000001_create_providers_table'
2026-07-07T20:21:35.825678148+00:00 INFO sea_orm_migration::migrator: Migration 'm20260520_000001_create_providers_table' has been applied
2026-07-07T20:21:35.826844491+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260520_000002_create_score_configs_table'
2026-07-07T20:21:35.832279347+00:00 INFO sea_orm_migration::migrator: Migration 'm20260520_000002_create_score_configs_table' has been applied
2026-07-07T20:21:35.833349348+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260520_000003_create_scorers_table'
2026-07-07T20:21:35.839950404+00:00 INFO sea_orm_migration::migrator: Migration 'm20260520_000003_create_scorers_table' has been applied
2026-07-07T20:21:35.841036127+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260520_000004_create_online_eval_jobs_table'
2026-07-07T20:21:35.845749960+00:00 INFO sea_orm_migration::migrator: Migration 'm20260520_000004_create_online_eval_jobs_table' has been applied
2026-07-07T20:21:35.846896727+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260520_000005_drop_eval_templates_table'
2026-07-07T20:21:35.847658968+00:00 INFO sea_orm_migration::migrator: Migration 'm20260520_000005_drop_eval_templates_table' has been applied
2026-07-07T20:21:35.848715252+00:00 INFO sea_orm_migration::migrator: Applying migration 'm20260604_000001_add_kind_to_pipeline'
2026-07-07T20:21:35.849756777+00:00 INFO sea_orm_migration::migrator: Migration 'm20260604_000001_add_kind_to_pipeline' has been applied
2026-07-07T20:21:35.851627502+00:00 INFO openobserve::migration: DB upgrade completed to version 45
2026-07-07T20:21:35.852358160+00:00 INFO infra::cache::file_data::disk: Loading disk cache start
2026-07-07T20:21:35.852382026+00:00 INFO infra::cache::file_data::disk: [disk_cache_gc] pausable job started
2026-07-07T20:21:35.852541842+00:00 INFO infra::cache::file_data::memory: [memory_cache_gc] pausable job started
2026-07-07T20:21:35.852891373+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_start_dt_idx on table meta
2026-07-07T20:21:35.852925734+00:00 INFO infra::cache::file_data::disk: Loading disk cache done, total files: 0
2026-07-07T20:21:35.853154372+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_start_dt_idx created successfully
2026-07-07T20:21:35.853180933+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_idx on table meta
2026-07-07T20:21:35.853390298+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_idx created successfully
2026-07-07T20:21:35.853418362+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_key1_idx on table meta
2026-07-07T20:21:35.853717785+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_key1_idx created successfully
2026-07-07T20:21:35.853741611+00:00 INFO infra::db::sqlite: [SQLITE] creating index meta_module_start_dt_idx on table meta
2026-07-07T20:21:35.853947105+00:00 INFO infra::db::sqlite: [SQLITE] index meta_module_start_dt_idx created successfully
2026-07-07T20:21:35.860863098+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_org_idx on table file_list
2026-07-07T20:21:35.861099243+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_org_idx created successfully
2026-07-07T20:21:35.861123168+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_stream_ts_idx on table file_list
2026-07-07T20:21:35.861342927+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_stream_ts_idx created successfully
2026-07-07T20:21:35.861369127+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_stream_date_idx on table file_list
2026-07-07T20:21:35.861601993+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_stream_date_idx created successfully
2026-07-07T20:21:35.861633323+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_updated_at_deleted_idx on table file_list
2026-07-07T20:21:35.861862863+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_updated_at_deleted_idx created successfully
2026-07-07T20:21:35.861891488+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_history_org_idx on table file_list_history
2026-07-07T20:21:35.862103170+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_history_org_idx created successfully
2026-07-07T20:21:35.862126064+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_history_stream_ts_idx on table file_list_history
2026-07-07T20:21:35.862359324+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_history_stream_ts_idx created successfully
2026-07-07T20:21:35.862387218+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_deleted_created_at_idx on table file_list_deleted
2026-07-07T20:21:35.862654811+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_deleted_created_at_idx created successfully
2026-07-07T20:21:35.862680120+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_deleted_stream_date_file_idx on table file_list_deleted
2026-07-07T20:21:35.862889942+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_deleted_stream_date_file_idx created successfully
2026-07-07T20:21:35.862915741+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_jobs_stream_status_idx on table file_list_jobs
2026-07-07T20:21:35.863135682+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_jobs_stream_status_idx created successfully
2026-07-07T20:21:35.863163154+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_jobs_status_dumped_idx on table file_list_jobs
2026-07-07T20:21:35.863386534+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_jobs_status_dumped_idx created successfully
2026-07-07T20:21:35.863410099+00:00 INFO infra::db::sqlite: [SQLITE] creating index stream_stats_org_idx on table stream_stats
2026-07-07T20:21:35.863656386+00:00 INFO infra::db::sqlite: [SQLITE] index stream_stats_org_idx created successfully
2026-07-07T20:21:35.863690171+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_dump_stats_org_idx on table file_list_dump_stats
2026-07-07T20:21:35.863931778+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_dump_stats_org_idx created successfully
2026-07-07T20:21:35.863959160+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_history_stream_file_idx on table file_list_history
2026-07-07T20:21:35.864188260+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_history_stream_file_idx created successfully
2026-07-07T20:21:35.864214089+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_jobs_stream_offsets_idx on table file_list_jobs
2026-07-07T20:21:35.864427007+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_jobs_stream_offsets_idx created successfully
2026-07-07T20:21:35.864451073+00:00 INFO infra::db::sqlite: [SQLITE] creating index stream_stats_org_stream_recent_idx on table stream_stats
2026-07-07T20:21:35.864661467+00:00 INFO infra::db::sqlite: [SQLITE] index stream_stats_org_stream_recent_idx created successfully
2026-07-07T20:21:35.864695712+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_dump_stats_stream_file_idx on table file_list_dump_stats
2026-07-07T20:21:35.864918308+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_dump_stats_stream_file_idx created successfully
2026-07-07T20:21:35.864951542+00:00 INFO infra::db::sqlite: [SQLITE] creating index file_list_stream_file_idx on table file_list
2026-07-07T20:21:35.865159731+00:00 INFO infra::db::sqlite: [SQLITE] index file_list_stream_file_idx created successfully
2026-07-07T20:21:36.865649312+00:00 INFO infra: Shutting down DDL connection pool
2026-07-07T20:21:36.867140389+00:00 INFO openobserve::common::infra: Generating new instance id
2026-07-07T20:21:37.869344787+00:00 INFO ingester: Start ingester init
2026-07-07T20:21:37.870316481+00:00 INFO ingester::wal: Scanning lock files from "/data/wal/logs"
2026-07-07T20:21:37.870863153+00:00 INFO ingester::wal: Found 0 lock files
2026-07-07T20:21:37.871397017+00:00 INFO ingester::wal: Check uncompleted parquet files done
2026-07-07T20:21:37.871461310+00:00 INFO ingester: Ingesters init done
2026-07-07T20:21:37.871526396+00:00 INFO ingester: Scanning wal files from "/data/wal/logs"
2026-07-07T20:21:37.871778408+00:00 INFO ingester: Found 0 wal files to replay
2026-07-07T20:21:37.871820629+00:00 INFO ingester: Replay wal files done
2026-07-07T20:21:37.897719346+00:00 INFO openobserve::service::enrichment_table::url_processor: [ENRICHMENT::URL] URL job processor initialized
2026-07-07T20:21:37.897807218+00:00 INFO openobserve::service::enrichment_table::url_processor: [ENRICHMENT::URL] Starting URL job processor
2026-07-07T20:21:37.897839821+00:00 INFO openobserve::service::enrichment_table::url_processor: [ENRICHMENT::URL] Starting stale job recovery task
2026-07-07T20:21:37.898122674+00:00 INFO openobserve::service::db::user: Users Cached
2026-07-07T20:21:37.898451595+00:00 INFO openobserve::service::db::organization: Organizations Cached
2026-07-07T20:21:37.899178860+00:00 INFO openobserve::service::db::org_users: Organizations users Cached
2026-07-07T20:21:37.899483013+00:00 INFO openobserve::job::mmdb_downloader: Checking mmdb files
2026-07-07T20:21:37.899538428+00:00 INFO openobserve::service::db::org_ingestion_tokens: Org ingestion tokens cached: 1
2026-07-07T20:21:37.899860988+00:00 INFO openobserve::service::db::organization: Organization settings Cached
2026-07-07T20:21:37.899944398+00:00 INFO openobserve::service::db::org_users: Start watching org_users
2026-07-07T20:21:37.899950936+00:00 INFO openobserve::service::db::user: Start watching user
2026-07-07T20:21:37.900012339+00:00 INFO openobserve::service::db::org_ingestion_tokens: Start watching org_ingestion_tokens
2026-07-07T20:21:37.900069217+00:00 INFO openobserve::service::db::organization: Start watching organizations
2026-07-07T20:21:37.903031339+00:00 INFO openobserve::job: [telemetry] pausable job started
2026-07-07T20:21:37.903054494+00:00 INFO openobserve::service::db::short_url: Start watching short URLs
2026-07-07T20:21:37.903347127+00:00 INFO openobserve::service::db::short_url: [SHORT_URLS] Cached with len: 0
2026-07-07T20:21:37.903504978+00:00 INFO openobserve::service::db::schema: [Schema:watch] Start watching stream schema
2026-07-07T20:21:37.903531148+00:00 INFO openobserve::service::db::functions: Start watching function
2026-07-07T20:21:37.903545456+00:00 INFO openobserve::service::db::compact::retention: Start watching compact deleting
2026-07-07T20:21:37.903549193+00:00 INFO openobserve::service::db::metrics: Start watching prometheus cluster leader
2026-07-07T20:21:37.903553130+00:00 INFO openobserve::service::db::system_settings: Start watching system settings
2026-07-07T20:21:37.903619998+00:00 INFO openobserve::service::db::model_pricing: [model_pricing] start watching
2026-07-07T20:21:37.903629627+00:00 INFO openobserve::service::db::alerts::templates: Start watching alert templates
2026-07-07T20:21:37.903661538+00:00 INFO openobserve::service::db::alerts::destinations: Start watching alert destinations
2026-07-07T20:21:37.903671036+00:00 INFO openobserve::service::db::alerts::realtime_triggers: Start watching alert realtime triggers
2026-07-07T20:21:37.903686225+00:00 INFO infra::coordinator::alerts: Start watching alerts
2026-07-07T20:21:37.903691505+00:00 INFO openobserve::service::db::pipeline: [Pipeline::watch] His watch is started
2026-07-07T20:21:37.903694261+00:00 INFO openobserve::service::db::dashboards: [Dashboard::watch_id_to_org] started
2026-07-07T20:21:37.903707186+00:00 INFO openobserve::service::db::organization: Start watching organization settings
2026-07-07T20:21:37.903703829+00:00 INFO openobserve::service::db::enrichment_table: Start watching stream enrichment_table
2026-07-07T20:21:37.904064571+00:00 INFO openobserve::service::db::schema: Cache schema got 0 items
2026-07-07T20:21:37.904090521+00:00 INFO openobserve::service::db::schema: Stream schemas Cached 0 schemas
2026-07-07T20:21:37.904093506+00:00 INFO openobserve::service::db::schema: Stream schemas Cached 0 streams
2026-07-07T20:21:37.909569561+00:00 INFO openobserve::service::db::functions: Functions Cached
2026-07-07T20:21:37.910077796+00:00 INFO openobserve::service::db::metrics: Prometheus cluster leaders Cached
2026-07-07T20:21:37.910488754+00:00 INFO openobserve::service::db::system_settings: System settings cached
2026-07-07T20:21:37.913100957+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_slack' in default
2026-07-07T20:21:37.915351074+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_msteams' in default
2026-07-07T20:21:37.917213073+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_pagerduty' in default
2026-07-07T20:21:37.919019069+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_discord' in default
2026-07-07T20:21:37.920814794+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_webhook' in default
2026-07-07T20:21:37.922690961+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_opsgenie' in default
2026-07-07T20:21:37.924499530+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_servicenow' in default
2026-07-07T20:21:37.926579893+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] Created system template 'prebuilt_email' in default
2026-07-07T20:21:37.926607506+00:00 INFO openobserve::service::alerts::templates: [TEMPLATES] System templates initialization complete: 8 created, 0 updated, 0 already existed
2026-07-07T20:21:37.926965720+00:00 INFO openobserve::service::db::alerts::templates: 8 Templates Cached
2026-07-07T20:21:37.927485778+00:00 INFO openobserve::service::db::alerts::destinations: 0 destinations Cached
2026-07-07T20:21:37.927767569+00:00 INFO openobserve::service::db::alerts::realtime_triggers: Alert realtime triggers Cached
2026-07-07T20:21:37.927793478+00:00 INFO openobserve::service::db::alerts::alert: Alerts Cached
2026-07-07T20:21:37.927830780+00:00 INFO openobserve::job::pipeline_error_cleanup: [PIPELINE_ERROR_CLEANUP] Job initialized on ingester node
2026-07-07T20:21:37.927833124+00:00 INFO openobserve::job::session_cleanup: [SESSION_CLEANUP] Job initialized on ingester node with interval: 3600 seconds
2026-07-07T20:21:37.927836200+00:00 INFO openobserve::job: Job initialization complete
2026-07-07T20:21:37.927847672+00:00 INFO openobserve::service::runtime_metrics: Registered runtime 'job' for metrics collection
2026-07-07T20:21:37.927922856+00:00 INFO openobserve::job::compactor: [COMPACTOR::JOB] Compactor is enabled
2026-07-07T20:21:37.927931519+00:00 INFO openobserve: backend job init success
2026-07-07T20:21:37.927949237+00:00 INFO openobserve::job::promql: [promql_metrics_leader] pausable job started
2026-07-07T20:21:37.927958905+00:00 INFO openobserve::job::pipeline_error_cleanup: [pipeline_error_cleanup] pausable job started
2026-07-07T20:21:37.927962142+00:00 INFO openobserve::job::session_cleanup: [session_cleanup] pausable job started
2026-07-07T20:21:37.927971079+00:00 INFO openobserve::service::promql::search::cache: Loading disk metrics cache start
2026-07-07T20:21:37.927974185+00:00 INFO openobserve::service::promql::search::cache: Loading disk metrics cache done, total items: 0
2026-07-07T20:21:37.927989013+00:00 INFO openobserve::job::alert_manager: [alert_dedup_cleanup] pausable job started
2026-07-07T20:21:37.928032503+00:00 INFO openobserve::job::alert_manager: [alert_manager_watch_timeout] pausable job started
2026-07-07T20:21:37.928213594+00:00 INFO openobserve::job::compactor: [COMPACTOR::JOB] Running enrichment table merge
2026-07-07T20:21:37.928266806+00:00 INFO openobserve::job::compactor: [compactor_clean_done_jobs] pausable job started
2026-07-07T20:21:37.928298269+00:00 INFO openobserve::job::compactor: [run_delay_deletion] pausable job started
2026-07-07T20:21:37.928304671+00:00 INFO openobserve::job::compactor: [run_generate_old_data_job] pausable job started
2026-07-07T20:21:37.928307076+00:00 INFO openobserve::job::compactor: [run_merge] pausable job started
2026-07-07T20:21:37.928309210+00:00 INFO openobserve::job::compactor: [run_retention] pausable job started
2026-07-07T20:21:37.928314019+00:00 INFO openobserve::job::compactor: [compactor_sync_to_db] pausable job started
2026-07-07T20:21:37.928315722+00:00 INFO openobserve::job::compactor: [run_compactor_pending_jobs_metric] pausable job started
2026-07-07T20:21:37.928272087+00:00 INFO openobserve::job::compactor: [compactor_check_running_jobs] pausable job started
2026-07-07T20:21:37.928423066+00:00 INFO openobserve::job::stats: [cache_stream_stats] pausable job started
2026-07-07T20:21:37.928467902+00:00 INFO openobserve::job::stats: [file_list_update_stats] pausable job started
2026-07-07T20:21:37.928262848+00:00 INFO openobserve::job::compactor: [run_generate_job] pausable job started
2026-07-07T20:21:37.930108782+00:00 INFO openobserve::service::enrichment::storage::remote: [ENRICHMENT::STORAGE] Running enrichment table merge job
2026-07-07T20:21:37.930202442+00:00 INFO openobserve::service::enrichment::storage::remote: [enchrichment_table_merge_job] pausable job started
2026-07-07T20:21:37.931655025+00:00 INFO openobserve::service::runtime_metrics: Registered runtime 'grpc' for metrics collection
2026-07-07T20:21:37.931705923+00:00 INFO openobserve: starting gRPC server at 0.0.0.0:5081
2026-07-07T20:21:37.931755027+00:00 INFO openobserve::service::runtime_metrics: Registered runtime 'http' for metrics collection
2026-07-07T20:21:37.931781508+00:00 INFO openobserve::service::runtime_metrics: Starting runtime metrics collector (basic mode - compile with --cfg tokio_unstable for detail)
2026-07-07T20:21:37.931835532+00:00 INFO openobserve::service::db::schema: EnrichmentTables Cached
2026-07-07T20:21:37.932117673+00:00 INFO openobserve::service::db::pipeline: [Pipeline] Cached id->org mapping for 0 pipelines
2026-07-07T20:21:37.932455191+00:00 INFO openobserve::service::db::pipeline: [Pipeline] Cached realtime pipelines: 0, scheduled pipelines: 0
2026-07-07T20:21:37.932862221+00:00 INFO openobserve::service::db::dashboards: [Dashboard] Cached id->org mapping for 0 dashboards
2026-07-07T20:21:37.933255428+00:00 INFO infra::scheduler: deleting scheduled job: QueryRecommendations, query_recommendations
2026-07-07T20:21:37.933884823+00:00 INFO openobserve: [QUERY_RECOMMENDATIONS] Setup the initial trigger.
2026-07-07T20:21:37.933906605+00:00 INFO openobserve: Starting HTTP server at: 0.0.0.0:5080
2026-07-07T20:21:37.933912015+00:00 INFO openobserve: HTTP client IP sources (in order, implicit ConnectInfo fallback): XEnvoyExternalAddress,XRealIp,RightmostXForwardedFor
2026-07-07T20:21:37.955789231+00:00 INFO openobserve::common::meta::telemetry: sending a track event OpenObserve - Starting server
2026-07-07T20:21:41.657735301+00:00 INFO openobserve::job::mmdb_downloader: Maxmind client initialized
I can now fire up a dashboard page
I can login with that “root@example.com” user and see the main dashboard
Windows logs
Let’s start with some basic Windows logs.
Once installed, it will show up in Windows services as “otel-collector”
I can now see some streams showing up in the dashboard
I can now see things like BytesCommitted
I can also just jump into Logs to see all the logs coming in
For instance, it caught that I intentionally stopped the Instana agent (forgot it was still running to be honest)
It’s easy to turn a search into a Dashboard. Here I fired up a “test” dashboard with a “test” graph based on bytes committed and hostname
Here is a graph of CPU load
Alerts
I went to configure alerts and noted that when I setup email, it complains that there are no SMTP destinations
However, at least in the UI, I see no place to set those
Based on this blog entry they have, it looks like I need to configure that as environment variable settings set at launch
ZO_SMTP_ENABLED=true \
ZO_SMTP_HOST="smtp.gmail.com" \
ZO_SMTP_PORT=587 \
ZO_SMTP_USER_NAME="your-gmail-address@gmail.com" \
ZO_SMTP_PASSWORD="your-app-password" \
ZO_SMTP_FROM_EMAIL="your-gmail-address@gmail.com" \
ZO_SMTP_ENCRYPTION="starttls" \
ZO_ROOT_USER_EMAIL="root@example.com" \
ZO_ROOT_USER_PASSWORD="Complexpass#123" \
./openobserve
Cloud
We can sign up to cloud.openobserve.ai
I’ll create an account using email
I now have an instance
I always like to know what I’m getting for a trial/time. Here we see we get 20 “credits” for free. Normal ingestion is $0.50/Gb and $0.01/Gb for queries
Let’s assume I even like this tool, that is a pretty large number considering base price on ingest Gb is $0.40 for New Relic, and $0.10/Gb for Datadog.
I’ll try setting up the Linux agent first
$ curl -O https://raw.githubusercontent.com/openobserve/agents/main/linux/install.sh && chmod +x install.sh && sudo ./install.sh https://api.openobserve.ai/api/xxxxxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3318 100 3318 0 0 20000 0 0
[sudo: authenticate] Password:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 0
100 69.68M 100 69.68M 0 0 2.90M 0 00:23 00:23 2.93M
Created symlink '/etc/systemd/system/multi-user.target.wants/otel-collector.service' → '/etc/systemd/system/otel-collector.service'.
Otel-collector service started!
I can now see the service is running
$ sudo service otel-collector status
● otel-collector.service - OpenTelemetry Collector
Loaded: loaded (/etc/systemd/system/otel-collector.service; enabled; preset: enabled)
Active: active (running) since Tue 2026-07-07 18:21:26 CDT; 1min 24s ago
Invocation: 3a801ef0762d4ec89d67f96d72406539
Main PID: 1059998 (otelcol-contrib)
Tasks: 23 (limit: 18078)
Memory: 280.4M (peak: 281M)
CPU: 1.846s
CGroup: /system.slice/otel-collector.service
├─1059998 /usr/local/bin/otelcol-contrib --config /etc/otel-config.yaml
└─1060017 journalctl --utc --output=json --follow --priority info --directory /var/log/journal
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).makeFingerprint
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.111.0/fileconsumer/file.go:172
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).makeReaders
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.111.0/fileconsumer/file.go:199
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).consume
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.111.0/fileconsumer/file.go:149
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).poll
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.111.0/fileconsumer/file.go:131
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).startPoller.func1
Jul 07 18:22:48 LuiGi otelcol-contrib[1059998]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.111.0/fileconsumer/file.go:101
I pretty quickly see some logs coming through
And we can click any log to see more details.
Let’s push a sample metric into OpenObserve
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/my_test_metrics/_json" \
-u "isaac@freshbrewed.science:o2oi_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
-H "Content-Type: application/json" \
-d '[
{
"__name__": "curl_sample_metric",
"value": 99.5,
"host": "local-test-machine",
"job": "manual_curl_test"
}
]'
{"code":200,"status":[{"name":"my_test_metrics","successful":1,"failed":0}]}
I can see the metric in the data sources, but cannot seem to query it
I tried the AI but it didn’t find any metrics despite telling it the name
Let’s try it
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" \
-u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" \
-H "Content-Type: application/json" \
-d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 99.5
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
I’ll add a few more
now look at a graph of metrics
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/my_test_metrics/_json" \
-u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" \
-H "Content-Type: application/json" \
-d '[
{
"__name__": "curl_sample_metric",
"value": 99.5,
"host": "local-test-machine",
"job": "manual_curl_test"
}
]'
{"code":200,"status":[{"name":"my_test_metrics","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/my_test_metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"value": 99.5,
"host": "local-test-machine",
"job": "manual_curl_test"
}
]'
{"code":200,"status":[{"name":"my_test_metrics","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/my_test_metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"value": 99.5,
"host": "local-test-machine",
"job": "manual_curl_test"
}
]'
{"code":200,"status":[{"name":"my_test_metrics","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/my_test_metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"value": 99.4,
"host": "local-test-machine",
"job": "manual_curl_test"
}
]'
{"code":200,"status":[{"name":"my_test_metrics","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" \
-u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" \
-H "Content-Type: application/json" \
-d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 99.5
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 99.2
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 99.1
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 99.1
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 99.9
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 80.0
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
$ curl -X POST "https://api.openobserve.ai/api/3GCCBifZPAHrAqi0tDoOwRj1A7A/ingest/metrics/_json" -u "isaac@freshbrewed.science:o2oi_ESzRSW9ar9MzrxpbgZYWsSRqIa163X8a" -H "Content-Type: application/json" -d '[
{
"__name__": "curl_sample_metric",
"__type__": "gauge",
"host": "local-test-machine",
"job": "manual_curl_test",
"value": 99.9
}
]'
{"code":200,"status":[{"name":"curl_sample_metric","successful":1,"failed":0}]}
I can now query and show it as a metrics:
Notifications
I can create a sample message but only send it to destinations in my org
Note: the name is not the name of the person, rather the name of the destination object
We could then use this in an alert destination
In particular I like the Summary which talks out the alert condition in plain language
Let’s create a new log alert that would trigger to the IsaacJohnson email destination but also create an incident
https://cloud.openobserve.ai/web/alerts?org_identifier=3GCCBifZPAHrAqi0tDoOwRj1A7A
I can now see an incident created
And those incidents also show up as stream:
This means we can see a stream of events of incidents
Which means if I were an SRE manager or lead, I might care to see our trend of incidents over time
OpenTelemetry
Let’s follow the guide they have on making a simple Python app that will query a local Ollama model
I can see traces coming in from python
Once we ingest these traces, we can see some of the details
We can see some of those attributes like tokens in and out as well as model
and we can see some metrics above the list of all the spans
Summary
I really thought I would find the pricing to be just too high as I just was fixating on the per/Gb ingestion price.
However, once I fed all the current pricing details for New Relic, Datadog, Groundcover and OpenObserve into Gemini Pro - which should have no actual bias, and looked at small and large deployments
Here is a recording of my using the Gemini created widget:
I was actually way wrong - in almost all cases, the moment you add users, OpenObserve really is the lowest cost option.
Let’s look at a reasonably small deployment - we could get all that we wanted in for $10/mo
Or a large deployment as I tried to recall from memory
All things are no equal, however. Even if OpenObserve beats the rest on price, we have to consider that Datadog Incident management has a full featured app with the ability to do phone calls. To accomplish similar with Groundcover or OpenObserve I would need to pair up with PagerDuty.
The other intangible is how long a smaller player can last. I wrote about Epsagon in 2020 which was bought by Cisco and shutdown. In 2021 I wrote about Lightstep which was picked up by ServiceNow and shuttered.
So there can be a risk one must consider. However, my friend, if I may take liberties with the term, wouldn’t go to a shady place so I trust their judgment.