데이터베이스 마이그레이션 참조

데이터베이스 마이그레이션 참조 (Reference for Database Migrations)

airflow db migrate를 실행할 때 수행되는 모든 데이터베이스 마이그레이션 목록이에요. 각 Airflow 버전 사이의 마이그레이션이 언제, 어떤 내용으로 실행되는지 파악하는 데 도움이 된답니다.

출처: 문서

본문

airflow db migrate를 실행할 때 실행되는 모든 데이터베이스 마이그레이션 목록이에요.

Warning

이 마이그레이션 세부 정보는 주로 특정 Airflow 버전 사이의 마이그레이션 중 언제, 어떤 종류의 마이그레이션이 실행될지 사용자에게 알려주기 위해 사용돼요. 의도는 "DB를 의식하는" 사용자가 마이그레이션을 분석하고 그것이 자신의 Airflow 데이터베이스에 미치는 영향에 대해 결론을 내릴 수 있게 하는 거예요. 그런 사용자들은 Airflow 내부 DB 구조가 어떻게 생겼는지 이해하기 위해 데이터베이스 ERD 스키마 문서도 살펴보고 싶을 거예요. 하지만 구조는 내부적인 것이며, 데이터를 검색하거나 수정하기 위해 DB에 직접 접근해서는 안 된다는 점을 알아야 해요 — 그렇게 하려면 REST API를 사용해야 해요.

Revision ID Revises ID Airflow Version Description
d2f4e1b3c5a7 (head) 9ff64e1c35d3 3.3.0 Add partition_date to asset_partition_dag_run.
9ff64e1c35d3 dd5f3a8e2b91 3.3.0 Add indexes on dag_run.created_dag_version_id and task_instance.dag_version_id.
dd5f3a8e2b91 c20871fbf23a 3.3.0 Add rollup_fingerprint to AssetPartitionDagRun and index partitioned_asset_key_log.asset_partition_dag_run_id.
c20871fbf23a c9d4e5f6a7b8 3.3.0 Add partition_mapper_info to DagModel.
c9d4e5f6a7b8 a7e6d4c3b2f1 3.3.0 Add allow_consumer_teams columns to task_outlet_asset_reference table.
a7e6d4c3b2f1 8812eb67b63c 3.3.0 Add connection_test_request table for the deferred connection-test workflow.
8812eb67b63c acc215baed80 3.3.0 Change Deadline interval to JSON.
acc215baed80 a1b2c3d4e5f6 3.3.0 Add team_name to trigger table.
a1b2c3d4e5f6 a7f3b2c1d4e5 3.3.0 Add version_data to dag_version.
a7f3b2c1d4e5 b8f3e4a1d2c9 3.3.0 Add access control columns to dag_schedule_asset_reference table.
b8f3e4a1d2c9 fde9ed84d07b 3.3.0 Add retry_delay_override and retry_reason to task_instance.
fde9ed84d07b 9fabad868fdb 3.3.0 Add task_state_store and asset_state_store tables.
9fabad868fdb a4c2d171ae18 3.3.0 Add timetable_periodic to DagModel.
a4c2d171ae18 1d6611b6ab7c 3.3.0 Add dag_result to XComModel.
1d6611b6ab7c 888b59e02a5b 3.2.0 Add bundle_name to callback table.
888b59e02a5b 6222ce48e289 3.2.0 Fix migration file ORM inconsistencies.
6222ce48e289 134de42d3cb0 3.2.0 Add partition fields to DagModel.
134de42d3cb0 e42d9fcd10d9 3.2.0 Add partition_key to backfill_dag_run.
e42d9fcd10d9 f8c9d7e6b5a4 3.2.0 Add allowed_run_types to dag.
f8c9d7e6b5a4 53ff648b8a26 3.2.0 Standardize UUID column format for non-PostgreSQL databases.
53ff648b8a26 a5a3e5eb9b8d 3.2.0 Add revoked_token table.
a5a3e5eb9b8d 55297ae24532 3.2.0 Make external_executor_id TEXT to allow for longer external_executor_ids.
55297ae24532 e79fc784f145 3.2.0 Add required fields to enable UI integrations for the Deadline Alerts feature.
e79fc784f145 0b112f49112d 3.2.0 Add timetable_type to dag table for filtering.
0b112f49112d c47f2e1ab9d4 3.2.0 Add exceeds max runs flag to dag model.
c47f2e1ab9d4 edc4f85a4619 3.2.0 Add queue column to trigger table.
edc4f85a4619 b12d4f98a91e 3.2.0 Enforce the new NOT NULL expectations for log.event and dag.is_stale.
b12d4f98a91e 665854ef0536 3.2.0 Drop id column from team table and make name the primary key.
665854ef0536 e812941398f4 3.2.0 Update ORM for asset partitioning.
e812941398f4 b87d2135fa50 3.2.0 Replace deadline's inline callback fields with foreign key to callback table.
b87d2135fa50 69ddce9a7247 3.2.0 Restructure callback table.
69ddce9a7247 5cc8117e9285 3.2.0 Add fail_fast column to dag table.
5cc8117e9285 1b2c3d4e5f6g 3.2.0 Add Human In the Loop Detail History table.
1b2c3d4e5f6g ab6dc0c82d0e 3.2.0 Add length to dag_bundle_team.dag_bundle_name.
ab6dc0c82d0e 15d84ca19038 3.2.0 Change serialized_dag data column to JSONB for PostgreSQL.
15d84ca19038 509b94a1042d 3.2.0 Replace asset_trigger table with asset_watcher.
509b94a1042d 82dbd68e6171 3.1.8 Change signed_url_template from VARCHAR(200) to TEXT.
82dbd68e6171 cc92b33c6709 3.1.8 Add composite index (ti_id, id DESC) to task_reschedule.
cc92b33c6709 eaf332f43c7c 3.1.0 Add backward compatibility for serialized DAG format v3 to v2.
eaf332f43c7c a3c7f2b18d4e 3.1.0 Add last_parse_duration to dag model.
a3c7f2b18d4e 7582ea3f3dd5 3.1.0 Add tables to store teams and associations with dag bundles.
7582ea3f3dd5 a169942745c2 3.1.0 Make bundle_name not nullable.
a169942745c2 808787349f22 3.1.0 Remove dag_id from Deadline.
808787349f22 3bda03debd04 3.1.0 Modify deadline's callback schema.
3bda03debd04 f56f68b9e02f 3.1.0 Add url template and template params to DagBundleModel.
f56f68b9e02f 09fa89ba1710 3.1.0 Add callback_state to deadline.
09fa89ba1710 40f7c30a228b 3.1.0 Add trigger_id to deadline.
40f7c30a228b ffdb0566c7c0 3.1.0 Add Human In the Loop Detail table.
ffdb0566c7c0 66a7743fe20e 3.1.0 Add dag_favorite table.
66a7743fe20e 583e80dfcef4 3.1.0 Add triggering user to dag_run.
583e80dfcef4 3ac9e5732b1f 3.1.0 Add task_inlet_asset_reference table.
3ac9e5732b1f 0242ac120002 3.1.0 Change the on-delete behaviour of task_instance.dag_version_id foreign key constraint to RESTRICT.
0242ac120002 dfee8bd5d574 3.1.0 Rename Deadline column in the Deadline table from deadline to deadline_time and change its type from DateTime to UTC DateTime.
dfee8bd5d574 fe199e1abd77 3.1.0 Add Deadline to Dag.
fe199e1abd77 29ce7909c52b 3.0.3 Delete import errors.
29ce7909c52b 959e216a3abb 3.0.0 Change TI table to have unique UUID id/pk per attempt.
959e216a3abb 0e9519b56710 3.0.0 Rename is_active to is_stale column in dag table.
0e9519b56710 ec62e120484d 3.0.0 Rename run_type from 'dataset_triggered' to 'asset_triggered' in dag_run table.
ec62e120484d be2cc2f742cf 3.0.0 Add new otel span fields.
be2cc2f742cf d469d27e2a64 3.0.0 Support bundles in DagPriorityParsingRequest.
d469d27e2a64 16f7f5ee874e 3.0.0 Use ti_id as FK to TaskReschedule.
16f7f5ee874e cf87489a35df 3.0.0 Remove dag.default_view column.
cf87489a35df 7645189f3479 3.0.0 Use TI.id as primary key to TaskInstanceNote.
7645189f3479 e00344393f31 3.0.0 Add try_id to TI and TIH.
e00344393f31 6a9e7a527a88 3.0.0 Remove external_trigger field.
6a9e7a527a88 33b04e4bfa19 3.0.0 Add DagRun run_after.
33b04e4bfa19 8ea135928435 3.0.0 Add new task_instance field scheduled_dttm.
8ea135928435 e39a26ac59f6 3.0.0 Add relative fileloc column.
e39a26ac59f6 38770795785f 3.0.0 Remove pickled data from dagrun table.
38770795785f 5c9c0231baa2 3.0.0 Add asset reference models.
5c9c0231baa2 237cef8dfea1 3.0.0 Remove processor_subdir.
237cef8dfea1 038dc8bc6284 3.0.0 Add deadline alerts table.
038dc8bc6284 e229247a6cb1 3.0.0 Update trigger_timeout column in task_instance table to UTC.
e229247a6cb1 eed27faa34e3 3.0.0 Add DagBundleModel.
eed27faa34e3 9fc3fc5de720 3.0.0 Remove pickled data from xcom table.
9fc3fc5de720 2b47dc6bc8df 3.0.0 Add references between assets and triggers.
2b47dc6bc8df d03e4a635aa3 3.0.0 Add dag versioning.
d03e4a635aa3 d8cd3297971e 3.0.0 Drop DAG pickling.
d8cd3297971e 5f57a45b8433 3.0.0 Add last_heartbeat_at directly to TI.
5f57a45b8433 486ac7936b78 3.0.0 Drop task_fail table.
486ac7936b78 d59cbbef95eb 3.0.0 Remove scheduler_lock column.
d59cbbef95eb 05234396c6fc 3.0.0 Add UUID primary key to task_instance table.
05234396c6fc 3a8972ecb8f9 3.0.0 Rename dataset as asset.
3a8972ecb8f9 fb2d4922cd79 3.0.0 Add exception_reason and logical_date to BackfillDagRun.
fb2d4922cd79 5a5d66100783 3.0.0 Tweak AssetAliasModel to match AssetModel after AIP-76.
5a5d66100783 c3389cd7793f 3.0.0 Add AssetActive to track orphaning instead of a flag.
c3389cd7793f 0d9e73a75ee4 3.0.0 Add backfill to dag run model.
0d9e73a75ee4 44eabb1904b4 3.0.0 Add name and group fields to DatasetModel.
44eabb1904b4 16cbcb1c8c36 3.0.0 Update dag_run_note.user_id and task_instance_note.user_id columns to String.
16cbcb1c8c36 522625f6d606 3.0.0 Remove redundant index.
522625f6d606 1cdc775ca98f 3.0.0 Add tables for backfill.
1cdc775ca98f a2c32e6c7729 3.0.0 Rename execution_date to logical_date.
a2c32e6c7729 0bfc26bc256e 3.0.0 Add triggered_by field to DagRun.
0bfc26bc256e d0f1c55954fa 3.0.0 Rename DagModel schedule_interval to timetable_summary.
d0f1c55954fa 044f740568ec 3.0.0 Remove SubDAGs: is_subdag & root_dag_id columns from DAG table.
044f740568ec 5f2621c13b39 3.0.0 Drop ab_user.id foreign key.
5f2621c13b39 22ed7efa9da2 2.10.3 Rename dag_schedule_dataset_alias_reference constraint names.
22ed7efa9da2 8684e37832e6 2.10.0 Add dag_schedule_dataset_alias_reference table.
8684e37832e6 41b3bc7c0272 2.10.0 Add dataset_alias_dataset association table.
41b3bc7c0272 ec3471c1e067 2.10.0 Add try_number to audit log.
ec3471c1e067 05e19f3176be 2.10.0 Add dataset_alias_dataset_event.
05e19f3176be d482b7261ff9 2.10.0 Add dataset_alias.
d482b7261ff9 c4602ba06b4b 2.10.0 Add task_instance_history.
c4602ba06b4b 677fdbb7fc54 2.10.0 Add DagPriorityParsingRequest table.
677fdbb7fc54 0fd0c178cbe8 2.10.0 Add new executor field to db.
0fd0c178cbe8 686269002441 2.10.0 Add indexes on dag_id column in referencing tables.
686269002441 bff083ad727d 2.9.2 Fix inconsistency between ORM and migration files.
bff083ad727d 1949afb29106 2.9.2 Remove idx_last_scheduling_decision index on last_scheduling_decision in dag_run table.
1949afb29106 ee1467d4aa35 2.9.0 Update trigger kwargs type and encrypt.
ee1467d4aa35 b4078ac230a1 2.9.0 Add display name for dag and task instance.
b4078ac230a1 8e1c784a4fc7 2.9.0 Change value column type to longblob in xcom table for mysql.
8e1c784a4fc7 ab34f260b71c 2.9.0 Add max_consecutive_failed_dag_runs column to dag_model table.
ab34f260b71c d75389605139 2.9.0 Add dataset_expression in DagModel.
d75389605139 1fd565369930 2.9.0 Add run_id to (Audit) log table and increase event name length.
1fd565369930 88344c1d9134 2.9.0 Add rendered_map_index to TaskInstance.
88344c1d9134 10b52ebd31f7 2.8.1 Drop unused TI index.
10b52ebd31f7 bd5dfbe21f88 2.8.0 Add processor_subdir to ImportError.
bd5dfbe21f88 f7bf2a57d0a6 2.8.0 Make connection login/password TEXT.
f7bf2a57d0a6 375a816bbbf4 2.8.0 Add owner_display_name to (Audit) Log table.
375a816bbbf4 405de8318b3a 2.8.0 Add new field 'clear_number' to dagrun.
405de8318b3a 788397e78828 2.7.0 Add include_deferred column to pool.
788397e78828 937cbd173ca1 2.7.0 Add custom_operator_name column.
937cbd173ca1 4bc4d934e2bc 2.7.0 Add index to task_instance table.
4bc4d934e2bc (base) None 2.6.2 Create initial database state from Airflow v2.6.2.

더 알아보기 (Learn more)