The Graph
Investigating - [FIRING:1] DatasourceNoData Production (partial_outage indexers c448b23e-d637-4e71-a420-d83567efd033 P2-High A Polygon-zkEVM: Block ingestor lagging behind true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: [no value]
Labels:
- alertname = DatasourceNoData
- cmp = partial_outage
- component = indexers
- datasource_uid = c448b23e-d637-4e71-a420-d83567efd033
- grafana_folder = Production
- priority = P2-High
- ref_id = A
- rulename = Polygon-zkEVM: Block ingestor lagging behind
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Polygon-zkEVM: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/fdjeivgbjdm2oe/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceNoData&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=datasource_uid%3Dc448b23e-d637-4e71-a420-d83567efd033&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=ref_id%3DA&matcher=rulename%3DPolygon-zkEVM%3A+Block+ingestor+lagging+behind&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 10, 2024 - 05:28 UTC
Investigating - You might be getting warning in the logs saying

"Trying again after ipfs.cat failed (attempt #379) with result Err(Request(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv4(10.0.7.168)), port: None, path: "/api/v0/cat", query: Some("arg=Qm..."), fragment: None }, source: TimedOut }))"

We are currently investigating this issue.

May 10, 2024 - 05:02 UTC
Update - [Comment from Opsgenie]ruslan acknowledged alert: "[Grafana]: Boba: Block ingestor lagging behind"
May 08, 2024 - 09:27 UTC
Investigating - [FIRING:1] Boba: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Boba: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Boba: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/fdjeivhz9g7pca/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBoba%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 08, 2024 - 09:27 UTC
Monitoring - A fix has been implemented and we are monitoring the results.
Apr 30, 2024 - 17:06 UTC
Identified - Several users are reporting issues with files failing to upload to IPFS. The team is aware and working on a fix. We will provide an update when this is resolved.
Apr 27, 2024 - 01:47 UTC
Monitoring - Update on delegation balances in Explorer: the correct GRT amount should now be displaying. We will continue monitoring.
Apr 27, 2024 - 16:12 UTC
Update - We are continuing to work on a fix for this issue.
Apr 24, 2024 - 21:50 UTC
Identified - The issue has been identified and a fix is being implemented.
Apr 24, 2024 - 21:49 UTC

About This Site

Status, incident and maintenance information for The Graph.
Use the RSS feed together with https://zapier.com/apps/rss to have updates sent straight to Slack or Discord.

Network - Gateway ? Operational
90 days ago
99.85 % uptime
Today
Network - Explorer ? Degraded Performance
90 days ago
100.0 % uptime
Today
Network - Subgraph Studio ? Degraded Performance
90 days ago
100.0 % uptime
Today
Network - Network Subgraph ? Operational
90 days ago
100.0 % uptime
Today
Hosted Service - Queries ? Operational
90 days ago
99.97 % uptime
Today
Hosted Service - Subgraph Health Operational
90 days ago
100.0 % uptime
Today
Hosted Service - Indexing Major Outage
90 days ago
92.67 % uptime
Today
Mainnet Operational
90 days ago
99.92 % uptime
Today
Sepolia Operational
90 days ago
100.0 % uptime
Today
Holesky Operational
90 days ago
100.0 % uptime
Today
Polygon (Matic) Operational
90 days ago
99.99 % uptime
Today
Polygon Amoy Operational
90 days ago
100.0 % uptime
Today
Polygon zkEvm Operational
90 days ago
99.76 % uptime
Today
Polygon zkEvm Testnet Operational
90 days ago
99.91 % uptime
Today
BSC Operational
90 days ago
100.0 % uptime
Today
BSC Chapel Operational
90 days ago
100.0 % uptime
Today
Avalanche Operational
90 days ago
99.18 % uptime
Today
Avalanche Fuji Operational
90 days ago
100.0 % uptime
Today
xDai Operational
90 days ago
100.0 % uptime
Today
Fantom Operational
90 days ago
100.0 % uptime
Today
Fantom Testnet Operational
90 days ago
100.0 % uptime
Today
Celo Operational
90 days ago
100.0 % uptime
Today
Celo Alfajores Operational
90 days ago
100.0 % uptime
Today
Fuse Operational
90 days ago
100.0 % uptime
Today
Moonbeam Operational
90 days ago
100.0 % uptime
Today
Moonriver Operational
90 days ago
100.0 % uptime
Today
Arbitrum One Operational
90 days ago
100.0 % uptime
Today
Arbitrum Sepolia Operational
90 days ago
100.0 % uptime
Today
Optimism Operational
90 days ago
100.0 % uptime
Today
Optimism Sepolia Operational
90 days ago
100.0 % uptime
Today
Aurora Operational
90 days ago
100.0 % uptime
Today
Aurora Testnet Operational
90 days ago
100.0 % uptime
Today
Arweave ? Operational
90 days ago
100.0 % uptime
Today
Cosmos ? Operational
90 days ago
0.88 % uptime
Today
Osmosis ? Operational
90 days ago
0.88 % uptime
Today
zkSync Era Operational
90 days ago
100.0 % uptime
Today
Zksync Era Sepolia Operational
90 days ago
100.0 % uptime
Today
Base Operational
90 days ago
100.0 % uptime
Today
Base Testnet Major Outage
90 days ago
44.55 % uptime
Today
NEAR Operational
90 days ago
100.0 % uptime
Today
Near Testnet Operational
90 days ago
100.0 % uptime
Today
Harmony Operational
90 days ago
100.0 % uptime
Today
Scroll Mainnet Operational
90 days ago
100.0 % uptime
Today
Scroll Sepolia Operational
90 days ago
100.0 % uptime
Today
Linea Mainnet Operational
90 days ago
100.0 % uptime
Today
Linea Sepolia Operational
90 days ago
100.0 % uptime
Today
Blast Mainnet Operational
90 days ago
100.0 % uptime
Today
Blast Testnet Operational
90 days ago
100.0 % uptime
Today
Sei Testnet Operational
90 days ago
100.0 % uptime
Today
X Layer Mainnet Operational
90 days ago
100.0 % uptime
Today
X Layer Sepolia Operational
90 days ago
100.0 % uptime
Today
Theta Testnet Operational
90 days ago
100.0 % uptime
Today
Astar ZkEVM Mainnet Operational
90 days ago
99.89 % uptime
Today
Astar ZkEVM Sepolia Operational
90 days ago
100.0 % uptime
Today
Etherlink Testnet Operational
90 days ago
96.31 % uptime
Today
Gnosis Operational
90 days ago
100.0 % uptime
Today
Gnosis Chiado Operational
90 days ago
100.0 % uptime
Today
Boba Operational
90 days ago
100.0 % uptime
Today
Mode Mainnet Operational
90 days ago
100.0 % uptime
Today
Mode Testnet Operational
90 days ago
100.0 % uptime
Today
Bitcoin Substreams Operational
90 days ago
100.0 % uptime
Today
Hosted Service - Miscellaneous Degraded Performance
90 days ago
99.85 % uptime
Today
Deployments ? Operational
90 days ago
99.99 % uptime
Today
IPFS Degraded Performance
90 days ago
100.0 % uptime
Today
Subgraph Logs ? Operational
90 days ago
100.0 % uptime
Today
Explorer ? Operational
90 days ago
100.0 % uptime
Today
thegraph.com ? Degraded Performance
90 days ago
100.0 % uptime
Today
The Graph Network Subgraph - Arbitrum Operational
90 days ago
95.49 % uptime
Today
Firehose Indexing Operational
90 days ago
100.0 % uptime
Today
Firehose Service Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Past Incidents
May 10, 2024
Resolved - This incident has been resolved.
May 10, 05:33 UTC
Update - [Comment from Opsgenie]Ruslan Rotaru acknowledged alert: "[Grafana]: Polygon-zkEVM: Block ingestor lagging behind"
May 10, 00:14 UTC
Investigating - [FIRING:1] Polygon-zkEVM: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Polygon-zkEVM: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Polygon-zkEVM: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/fdjeivgbjdm2oe/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DPolygon-zkEVM%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 10, 00:13 UTC
Resolved - This incident has been resolved.
May 10, 04:59 UTC
Investigating - When deploying your subgraph to `osmosis` / `cosmos` , you might see this warning

`
Unfortunately, proper support for Osmosis in the Hosted Service can not be guaranteed for the time being. Please refer to https://status.thegraph.com for more information. If this is affecting you in any way, please reach out to info+osmosis@thegraph.foundation.
`

We are currently investigating this issue.

Mar 8, 12:45 UTC
May 9, 2024

No incidents reported.

May 8, 2024
Resolved - This incident has been resolved.
May 8, 20:52 UTC
Update - [Comment from Opsgenie]Ruslan Rotaru acknowledged alert: "[Grafana]: Theta-Testnet-001: Block ingestor lagging behind"
May 8, 15:23 UTC
Investigating - [FIRING:1] Theta-Testnet-001: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Theta-Testnet-001: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Theta-Testnet-001: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivhuenbi8d/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DTheta-Testnet-001%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 8, 15:23 UTC
Resolved - This incident has been resolved.
May 8, 04:03 UTC
Update - [Comment from Opsgenie]ruslan acknowledged alert: "[Grafana]: Bitcoin: Block ingestor lagging behind"
May 8, 03:48 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0.21311475409836064
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

May 8, 03:38 UTC
May 7, 2024

No incidents reported.

May 6, 2024

No incidents reported.

May 5, 2024
Resolved - This incident has been resolved.
May 5, 12:08 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

May 5, 11:58 UTC
Resolved - This incident has been resolved.
May 5, 10:43 UTC
Update - [Comment from Opsgenie]Muhammad Perreira acknowledged alert: "[Grafana]: Bitcoin: Block ingestor lagging behind"
May 5, 10:33 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0.5573770491803278
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

May 5, 10:33 UTC
Resolved - This incident has been resolved.
May 5, 09:53 UTC
Investigating - [FIRING:3] DatasourceNoData Production (partial_outage indexers c448b23e-d637-4e71-a420-d83567efd033 P2-High A true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: [no value]
Labels:
- alertname = DatasourceNoData
- cmp = partial_outage
- component = indexers
- datasource_uid = c448b23e-d637-4e71-a420-d83567efd033
- grafana_folder = Production
- priority = P2-High
- ref_id = A
- rulename = Linea: Block ingestor lagging behind
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Linea: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/adjeivhrhrkzkb/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceNoData&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=datasource_uid%3Dc448b23e-d637-4e71-a420-d83567efd033&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=ref_id%3DA&matcher=rulename%3DLinea%3A+Block+ingestor+lagging+behind&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Value: [no value]
Labels:
- alertname = DatasourceNoData
- cmp = partial_outage
- component = indexers
- datasource_uid = c448b23e-d637-4e71-a420-d83567efd033
- grafana_folder = Production
- priority = P2-High
- ref_id = A
- rulename = Base-Sepolia: Block ingestor lagging behind
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Base-Sepolia: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/cdjeivf5alfk0a/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceNoData&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=datasource_uid%3Dc448b23e-d637-4e71-a420-d83567efd033&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=ref_id%3DA&matcher=rulename%3DBase-Sepolia%3A+Block+ingestor+lagging+behind&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Value: [no value]
Labels:
- alertname = DatasourceNoData
- cmp = partial_outage
- component = indexers
- datasource_uid = c448b23e-d637-4e71-a420-d83567efd033
- grafana_folder = Production
- priority = P2-High
- ref_id = A
- rulename = Bitcoin: Block ingestor lagging behind
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceNoData&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=datasource_uid%3Dc448b23e-d637-4e71-a420-d83567efd033&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=ref_id%3DA&matcher=rulename%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

May 5, 09:28 UTC
Resolved - This incident has been resolved.
May 5, 08:46 UTC
Investigating - [FIRING:2] DatasourceNoData Production (partial_outage indexers c448b23e-d637-4e71-a420-d83567efd033 P2-High A true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: [no value]
Labels:
- alertname = DatasourceNoData
- cmp = partial_outage
- component = indexers
- datasource_uid = c448b23e-d637-4e71-a420-d83567efd033
- grafana_folder = Production
- priority = P2-High
- ref_id = A
- rulename = Arbitrum-Sepolia: Block ingestor lagging behind
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Arbitrum-Sepolia: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivgjdk54wd/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceNoData&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=datasource_uid%3Dc448b23e-d637-4e71-a420-d83567efd033&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=ref_id%3DA&matcher=rulename%3DArbitrum-Sepolia%3A+Block+ingestor+lagging+behind&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Value: [no value]
Labels:
- alertname = DatasourceNoData
- cmp = partial_outage
- component = indexers
- datasource_uid = c448b23e-d637-4e71-a420-d83567efd033
- grafana_folder = Production
- priority = P2-High
- ref_id = A
- rulename = Fuse: Block ingestor lagging behind
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Fuse: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivhm0hlogc/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceNoData&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=datasource_uid%3Dc448b23e-d637-4e71-a420-d83567efd033&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=ref_id%3DA&matcher=rulename%3DFuse%3A+Block+ingestor+lagging+behind&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 5, 08:26 UTC
Resolved - This incident has been resolved.
May 5, 03:31 UTC
Investigating - [FIRING:1] DatasourceNoData Production (partial_outage indexers c448b23e-d637-4e71-a420-d83567efd033 P2-High A NEAR-Testnet: Block ingestor lagging behind true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: [no value]
Labels:
- alertname = DatasourceNoData
- cmp = partial_outage
- component = indexers
- datasource_uid = c448b23e-d637-4e71-a420-d83567efd033
- grafana_folder = Production
- priority = P2-High
- ref_id = A
- rulename = NEAR-Testnet: Block ingestor lagging behind
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = NEAR-Testnet: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/adjeivijl4gzkb/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceNoData&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=datasource_uid%3Dc448b23e-d637-4e71-a420-d83567efd033&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=ref_id%3DA&matcher=rulename%3DNEAR-Testnet%3A+Block+ingestor+lagging+behind&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 5, 03:11 UTC
May 4, 2024

No incidents reported.

May 3, 2024
Resolved - This incident has been resolved.
May 3, 13:58 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0.6666666666666666
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

May 3, 13:53 UTC
Resolved - This incident has been resolved.
May 3, 13:00 UTC
Update - [Comment from Opsgenie]Ruslan Rotaru acknowledged alert: "[Grafana]: Arbitrum-Sepolia: Block ingestor lagging behind"
May 2, 23:48 UTC
Investigating - [FIRING:1] Arbitrum-Sepolia: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Arbitrum-Sepolia: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Arbitrum-Sepolia: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivgjdk54wd/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DArbitrum-Sepolia%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 2, 23:21 UTC
Resolved - This incident has been resolved.
May 3, 02:02 UTC
Investigating - [FIRING:1] Polygon-Amoy: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Polygon-Amoy: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Polygon-Amoy: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivhcfvlz4a/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DPolygon-Amoy%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 3, 01:52 UTC
May 2, 2024
Resolved - This incident has been resolved.
May 2, 20:15 UTC
Update - [Comment from Opsgenie]Ruslan Rotaru acknowledged alert: "[Grafana]: Arbitrum-Sepolia: Block ingestor lagging behind"
May 2, 16:06 UTC
Investigating - [FIRING:1] Arbitrum-Sepolia: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Arbitrum-Sepolia: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Arbitrum-Sepolia: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivgjdk54wd/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DArbitrum-Sepolia%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

May 2, 15:06 UTC
May 1, 2024
Resolved - This incident has been resolved.
May 1, 22:33 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0.5901639344262295
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

May 1, 22:33 UTC
Apr 30, 2024
Resolved - This incident has been resolved.
Apr 30, 22:46 UTC
Investigating - [FIRING:1] Arbitrum-One: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Arbitrum-One: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Arbitrum-One: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivf9xwmpsf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DArbitrum-One%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Apr 30, 22:41 UTC
Resolved - This incident has been resolved.
Apr 30, 22:46 UTC
Investigating - [FIRING:1] Arbitrum-Sepolia: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Arbitrum-Sepolia: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Arbitrum-Sepolia: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivgjdk54wd/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DArbitrum-Sepolia%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Apr 30, 22:41 UTC
Resolved - This incident has been resolved.
Apr 30, 19:28 UTC
Update - [Comment from Opsgenie]Muhammad Perreira acknowledged alert: "[Grafana]: Bitcoin: Block ingestor lagging behind"
Apr 30, 18:59 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0, B1=0.65
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

Apr 30, 18:58 UTC
Apr 29, 2024

No incidents reported.

Apr 28, 2024
Resolved - This incident has been resolved.
Apr 28, 23:47 UTC
Update - [Comment from Opsgenie]ruslan acknowledged alert: "[Grafana]: Polygon-Amoy: Block ingestor lagging behind"
Apr 28, 23:45 UTC
Investigating - [FIRING:1] Polygon-Amoy: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Polygon-Amoy: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Polygon-Amoy: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/edjeivhcfvlz4a/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DPolygon-Amoy%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Apr 28, 22:37 UTC
Resolved - This incident has been resolved.
Apr 28, 14:07 UTC
Update - [Comment from Opsgenie]Ruslan Rotaru acknowledged alert: "[Grafana]: Bitcoin: Block ingestor lagging behind"
Apr 28, 13:54 UTC
Update - [Comment from Opsgenie]Ruslan Rotaru acknowledged alert: "[Grafana]: Bitcoin: Block ingestor lagging behind"
Apr 28, 11:53 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1
Dashboard: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1
Panel: https://thegraph.grafana.net/d/7rcuDImZk?orgId=1&viewPanel=150

Apr 28, 11:53 UTC
Resolved - This incident has been resolved.
Apr 28, 06:48 UTC
Update - [Comment from Opsgenie]ruslan acknowledged alert: "[Grafana]: Bitcoin: Block ingestor lagging behind"
Apr 28, 06:22 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Apr 28, 05:18 UTC
Apr 27, 2024

Unresolved incident: Delegated GRT amounts are not appearing on the website front-end.

Apr 26, 2024
Resolved - This incident has been resolved.
Apr 26, 05:53 UTC
Investigating - [FIRING:1] Bitcoin: Block ingestor lagging behind Production (partial_outage indexers P2-High true Infrastructure)
https://thegraph.grafana.net/alerting/list

**Firing**

Value: B0=0
Labels:
- alertname = Bitcoin: Block ingestor lagging behind
- cmp = partial_outage
- component = indexers
- grafana_folder = Production
- priority = P2-High
- statuspage = true
- team = Infrastructure
Annotations:
- description = Triggered when the block ingestor is lagging
- statuspage = Bitcoin: Block ingestor lagging behind
Source: https://thegraph.grafana.net/alerting/grafana/ddjeivfxbgf7kf/view?orgId=1
Silence: https://thegraph.grafana.net/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DBitcoin%3A+Block+ingestor+lagging+behind&matcher=cmp%3Dpartial_outage&matcher=component%3Dindexers&matcher=grafana_folder%3DProduction&matcher=priority%3DP2-High&matcher=statuspage%3Dtrue&matcher=team%3DInfrastructure&orgId=1

Apr 26, 01:03 UTC