Skip to content

Commit bab4e03

Browse files
committed
Add multi-file schema support design document.
1 parent 45b8826 commit bab4e03

File tree

1 file changed

+121
-0
lines changed

1 file changed

+121
-0
lines changed
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# Multi-File Schema Support
2+
3+
## Summary
4+
5+
Allow pgmold to load schema definitions from multiple SQL files, directories, and glob patterns instead of a single file.
6+
7+
## User Interface
8+
9+
The `--schema` argument accepts multiple sources. Each source can be:
10+
11+
- **File**: `sql:schema/users.sql`
12+
- **Directory**: `sql:schema/` (discovers all `*.sql` files recursively)
13+
- **Glob**: `sql:schema/**/*.sql`
14+
15+
Examples:
16+
17+
```bash
18+
# Single file (backwards compatible)
19+
pgmold plan --schema sql:schema.sql --database db:postgres://...
20+
21+
# Directory
22+
pgmold plan --schema sql:./schema/ --database db:postgres://...
23+
24+
# Multiple explicit files
25+
pgmold plan --schema sql:enums.sql --schema sql:tables.sql --database db:postgres://...
26+
27+
# Glob pattern
28+
pgmold plan --schema "sql:migrations/*.sql" --database db:postgres://...
29+
30+
# Mixed
31+
pgmold plan --schema sql:./schema/ --schema sql:extras/audit.sql --database db:postgres://...
32+
```
33+
34+
The `sql:` prefix remains required to distinguish from `db:` sources. Paths without wildcards that point to directories are treated as `**/*.sql` globs.
35+
36+
## Design Decisions
37+
38+
1. **Conflict handling**: Error immediately if the same object (table, enum, function) is defined in multiple files. Error message includes both file paths.
39+
40+
2. **File ordering**: Order doesn't matter. All files are parsed and merged into a single `Schema`, then the existing topological sort handles dependencies.
41+
42+
## Implementation
43+
44+
### New module: `src/parser/loader.rs`
45+
46+
Responsible for resolving paths and loading multiple files:
47+
48+
```rust
49+
pub fn load_schema_sources(sources: &[String]) -> Result<Schema> {
50+
let mut files: Vec<PathBuf> = Vec::new();
51+
52+
for source in sources {
53+
let path = source.strip_prefix("sql:").ok_or(...)?;
54+
files.extend(resolve_glob(path)?);
55+
}
56+
57+
let mut merged = Schema::new();
58+
for file in &files {
59+
let partial = parse_sql_file(file)?;
60+
merge_schema(&mut merged, partial, file)?;
61+
}
62+
63+
Ok(merged)
64+
}
65+
```
66+
67+
### Conflict detection
68+
69+
When inserting into `BTreeMap`, check if key exists:
70+
71+
- Tables: error if `merged.tables.contains_key(&table.name)`
72+
- Enums: error if `merged.enums.contains_key(&enum.name)`
73+
- Functions: error if `merged.functions.contains_key(&func.signature())`
74+
75+
Error format:
76+
77+
```
78+
Error: Duplicate table "users" defined in:
79+
- schema/users.sql
80+
- schema/tables.sql
81+
```
82+
83+
### CLI changes
84+
85+
Change `--schema` from `String` to `Vec<String>`:
86+
87+
```rust
88+
#[arg(long, required = true)]
89+
schema: Vec<String>,
90+
```
91+
92+
Split source parsing:
93+
94+
- `parse_source(source: &str)` - handles single `db:` source (unchanged)
95+
- `load_sql_sources(sources: &[String])` - new function for `sql:` sources
96+
97+
### Dependencies
98+
99+
Add `glob` crate for pattern matching.
100+
101+
## Testing
102+
103+
### Unit tests (`src/parser/loader.rs`)
104+
105+
1. `resolve_single_file` - single SQL file resolves to itself
106+
2. `resolve_directory` - directory returns all `*.sql` files recursively
107+
3. `resolve_glob_pattern` - glob matches expected files
108+
4. `merge_schemas_no_conflict` - two disjoint schemas merge correctly
109+
5. `merge_schemas_duplicate_table_errors` - duplicate table name produces error
110+
6. `merge_schemas_duplicate_enum_errors` - same for enums
111+
7. `merge_schemas_duplicate_function_errors` - same for functions
112+
113+
### Integration test
114+
115+
Create `tests/fixtures/multi_file/` with:
116+
117+
- `enums.sql` - enum definitions
118+
- `tables/users.sql` - users table
119+
- `tables/posts.sql` - posts table with FK to users
120+
121+
Test full flow: glob resolution → parsing → merging → diffing against test database.

0 commit comments

Comments
 (0)