openstatus.dev
Openstatus
www.openstatus.dev
1### A guide on how to migrate your tinybird datasource
2
3> What to do when you want to add/remove/update a column in your `datasource`.
4
5The `_migration` folder includes:
6
7- `ping_response__v4.datasource` which represents the `VERSION 4` of our
8 datasource
9- `ping_response.datasource` which has the upgraded schema and a new `VERSION 5`
10- `tb_backfill_populate.pipe` will fill the datasource with all the data until a
11 given timestamp
12- `tb_materialized_until_change_ingest.pipe` will fill the data from a given
13 timestamp
14
15```
16tb push _migration/ping_response.datasource
17tb push _migration/tb_materialized_until_change_ingest.pipe
18# after the given ts, it is time to run the backfill populate
19tb push _migration/tb_backfill_populate.pipe --populate --wait
20# after populate ends, it is time to remove the pipe
21tb pipe rm tb_backfill_populate --yes
22```
23
24Check if all the rows have been migrated:
25
26```
27tb pipe _migration/tb_datasource_union.pipe
28# after checking the result of the pipe
29tb pipe rm tb_datasource_union.pipe --yes
30```
31
32---
33
34Link to the [issue](https://github.com/openstatusHQ/openstatus/issues/278) from
35Gonzalo as reference.
36
37
38<!-- FIXME: more content -->
39
40```bash
41python3 -m venv .venv
42source .venv/bin/activate
43pip install tinybird-cli
44tb auth -i
45```
46
47```bash
48tb pull
49tb push aggregate_*.pipe --populate
50tb push endpoint_*.pipe
51...
52```