Update attachment package #735

joshua-journey-apps · 2025-10-07T10:35:17Z

This a port of the Swift, Dart and Kotlin attachment's API to the JS SDK.

This addresses the following issues #715 and #714 by adding the meta_data column to the table and persisting it in the attachments table.

It is currently still a work-in-progress so there are a couple things that still needs to be done:

changeset-bot · 2025-10-07T10:35:21Z

🦋 Changeset detected

Latest commit: bd6a5b9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages

Name	Type
@powersync/react-native	Minor
@powersync/common	Minor
@powersync/node	Minor
@powersync/web	Minor
@powersync/attachments	Patch
@powersync/adapter-sql-js	Patch
@powersync/op-sqlite	Patch
@powersync/tanstack-react-query	Patch
@powersync/diagnostics-app	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

stevensJourney

This looks like a good improvement so far. I've added some comments for some items I spotted.

packages/attachments/src/AttachmentContext.ts

packages/common/src/attachments/AttachmentContext.ts

packages/attachments/src/StorageService.ts

packages/common/src/attachments/SyncingService.ts

packages/attachments/src/StorageService.ts

packages/attachments/src/storageAdapters/NodeFileSystemAdapter.ts

packages/attachments/vitest.config.ts

packages/attachments/src/AttachmentQueue.ts

packages/common/src/attachments/AttachmentQueue.ts

…ile saving.

- Added `getAttachment` method to retrieve an attachment by ID in AttachmentContext. - Updated `upsertAttachment` to handle null values for optional fields. - Introduced `generateAttachmentId` method in AttachmentQueue for generating unique IDs. - Modified `watchActiveAttachments` to accept a throttle parameter. - Added `deleteFile` method to have attachment deletion. - Updated tests to cover new functionality and ensure reliability.

…ttachment sync operations - Added AttachmentErrorHandler interface to manage download, upload, and delete errors for attachments. - Updated AttachmentQueue and SyncingService to utilize the new error handler. - Enhanced tests to verify error handling behavior during attachment sync processes.

…ules

stevensJourney

Left some comments. Overall the APIs and code looks good so far.

demos/react-native-supabase-todolist/library/powersync/system.ts

packages/attachments/package.json

packages/common/src/attachments/AttachmentQueue.ts

packages/common/src/attachments/SyncingService.ts

packages/common/tests/attachments.test.ts

packages/react-native/package.json

stevensJourney

I noticed a few items in this review, but I do think this is looking good.

packages/common/src/attachments/AttachmentQueue.ts

packages/react-native/package.json

packages/react-native/rollup.config.mjs

packages/react-native/src/attachments/ReactNativeFileSystemAdapter.ts

…ger assignment

stevensJourney · 2025-12-29T07:51:30Z

packages/react-native/package.json

    "@rollup/plugin-replace": "^5.0.7",
    "@rollup/plugin-terser": "^0.4.4",
    "bson": "^6.10.4",
+    "expo-file-system": "^15.2.2",


While reviewing #803, I noticed that the latest version of expo-file-system is 19.0.21.

The current latest version uses a * peerDependency constraint for expo, which should theoretically be compatible with any version of Expo.

However, we're already 4 major versions behind expo-file-system, which raises an important question: what happens when the next major version (with breaking changes) is released? Bumping our peerDependency range for expo-file-system by a major version could be considered a breaking change for our users, requiring a major version bump of @powersync/react-native. I'd prefer to avoid making a major version bump solely for a filesystem dependency upgrade that only affects a small portion of this package's functionality (the attachment helpers).

We don't really have this issue for other JS platforms (or other SDKs) since the storage APIs (and general dependencies) we're using are quite standard.

Moving the FileSystem adapter out of this package is also not ideal, but does seem like a potentially more manageable alternative.

Good call, while not ideal, I'm not sure if there is a cleaner way to workaround the Expo dependency churn.

The lowest friction route I could think of, was to just create a straightforward@powersync/attachments-react-native package that handles both Expo and React Native file system adapters.

It's still a work-in-progress (docs and demos need updating) but the idea is there.

stevensJourney · 2026-01-05T11:46:35Z

packages/attachments-react-native/CHANGELOG.md

+
+### Features
+
+- Initial release


It's generally safer to add a minor changeset entry for this new package which contains this content.

While it's not the case here (since we have a changeset for the PowerSync SDKs). Merging this PR to main without a changeset, for a new package, could result in that package getting published immediately - while having a changeset would create a release PR where one can smoke check versions before release.

stevensJourney · 2026-01-05T11:49:00Z

packages/attachments-react-native/package.json

+  },
+  "peerDependencies": {
+    "@powersync/common": "workspace:^1.43.1",
+    "expo-file-system": ">=15.0.0 <20.0.0",


Is this version constraint correct? My understanding is that version 19 included breaking changes (compared to previous versions). How does our implementation here handle both APIs?

stevensJourney · 2026-01-05T11:59:18Z

packages/attachments-react-native/src/ExpoFileSystemStorageAdapter.ts

+import { decode as decodeBase64, encode as encodeBase64 } from 'base64-arraybuffer';
+import { AttachmentData, EncodingType, LocalStorageAdapter } from '@powersync/common';
+
+type ExpoFileSystemModule = {


Can't we import the type only from the package?

import type ExpoFS from 'expo-file-system' // ... export class ExpoFileSystemStorageAdapter implements LocalStorageAdapter { private fs: typeof ExpoFS;

One I made that change, and was using expo-file-system-19 I quickly ran into other issues due to the different API.

I'd suggest having this lib only support version 19.

stevensJourney · 2026-01-05T11:59:40Z

packages/attachments-react-native/src/ReactNativeFileSystemStorageAdapter.ts

+import { decode as decodeBase64, encode as encodeBase64 } from 'base64-arraybuffer';
+import { AttachmentData, EncodingType, LocalStorageAdapter } from '@powersync/common';
+
+type ReactNativeFsModule = {


I'd also recommend just doing type import here.

stevensJourney · 2026-01-05T12:32:33Z

packages/common/src/attachments/AttachmentQueue.ts

+  /** Service for managing attachment-related database operations */
+  readonly attachmentService: AttachmentService;
+
+  watchActiveAttachments: DifferentialWatchedQuery<AttachmentRecord>;


Should these, and watchAttachments not be private/protected?

stevensJourney · 2026-01-05T12:32:48Z

packages/common/src/attachments/AttachmentQueue.ts

+  ) => void;
+
+  /** Name of the database table storing attachment records */
+  readonly tableName?: string;


should this be optional/nullable?

stevensJourney · 2026-01-05T12:33:41Z

packages/common/src/attachments/AttachmentQueue.ts

+   * @returns Promise resolving to the new attachment ID
+   */
+  async generateAttachmentId(): Promise<string> {
+    return this.attachmentService.withContext(async (ctx) => {


Nit: this doesn't need an attachment context (since it can't possibly interfere with attachment sync). this can just use the DB.

stevensJourney · 2026-01-05T12:35:45Z

packages/common/src/attachments/AttachmentQueue.ts

+
+    // Sync storage periodically
+    this.#periodicSyncTimer = setInterval(async () => {
+      await this.syncStorage();


Also call this if the sync status changed to connected: true this allows for attachments to be processed as soon as the device becomes online (after being offline)

stevensJourney · 2026-01-05T12:40:40Z

packages/common/src/attachments/AttachmentQueue.ts

+  readonly downloadAttachments: boolean = true;
+
+  /** Maximum number of archived attachments to keep before cleanup. Default: 100 */
+  readonly archivedCacheLimit: number;


For consistency with other SDKs, I think a lot of these members can be protected - or, I don't think they need to be public.

stevensJourney · 2026-01-05T12:42:43Z

packages/common/src/attachments/SyncingService.ts

+  localStorage: LocalStorageAdapter;
+  remoteStorage: RemoteStorageAdapter;
+  logger: ILogger;
+  errorHandler?: AttachmentErrorHandler;


Also, a lot of these members don't need to be public. I don't imagine there's much of a use case for code of the form syncingService.localStorage. ... etc.

stevensJourney · 2026-01-05T14:27:59Z

packages/web/src/attachments/IndexDBFileSystemAdapter.ts

+        }
+
+        // Default base64 encoding
+        const base64String = req.result.replace(/^data:\w+;base64,/, '');


It looks like the saveFile method stores the data as an ArrayBuffer, where the req.result should be an ArrayBuffer. When reading a file, the replace method is not found on an ArrayBuffer.

joshua-journey-apps added 5 commits October 6, 2025 15:40

initial implementation

01070e1

Add make, remove and get uri endpoints to local storage

23dc8e4

Fix exporting node and index db storage adapters

ecb71cb

Wip add sync throttle & cache limiting

029df9d

Add downloading attachment test

958675a

joshua-journey-apps requested a review from stevensJourney October 7, 2025 10:35

stevensJourney reviewed Oct 21, 2025

View reviewed changes

joshuabrink and others added 17 commits October 27, 2025 15:49

Add user defined storage adapter path

45773aa

Refactor watch active observer into dedicated service

533dfab

Add temporal units to variable name

643289d

Rename storage -> syncing service

6840fb0

Add updateHook to save file

d8d4ad9

Use async onUpdate callback

9509aeb

Improve comments

9ac685c

Fix closing the watch active attachments listener

ce33bab

tests(WIP:) initial node setup and few bug fixes

704c2a8

tests: fixed SyncService to rely on LocalStorage and be agnostic to f…

ff25b2b

…ile saving.

feat: added archival management to AttachmentQueue via AttachmentContext

ec1a733

docs: update README.md to document usage

f6b2343

feat: added ExpoFileSystemAdapter to the attachments package

4cb5753

refactor: refactored react native demo to use new attachments API

7e40d26

fix: expo blob compatability and attachment row mapping

206bd5f

joshuabrink force-pushed the attachment-package-refactor branch from 671e23a to bea28d9 Compare November 12, 2025 09:15

fix: node exclusive exports

e42ae45

joshuabrink force-pushed the attachment-package-refactor branch from bea28d9 to e42ae45 Compare November 12, 2025 09:15

fix: linting, dead code and move active watch into start sync

4d5a1bc

khawarizmus mentioned this pull request Nov 19, 2025

docs: revamp the attachment documentation powersync-ja/powersync-docs#289

Open

8 tasks

Amine added 3 commits November 28, 2025 17:06

chore: update dependencies in pnpm-lock.yaml to latest versions

d179b9c

fix: update import paths in attachment-related files to use local mod…

f401bc0

…ules

chore: add changeset

d86799a

joshuabrink marked this pull request as ready for review November 28, 2025 13:10

khawarizmus requested a review from stevensJourney December 1, 2025 07:02

stevensJourney reviewed Dec 4, 2025

View reviewed changes

joshuabrink added 9 commits December 9, 2025 12:48

chore: update deprecation message

08e062f

fix: should retry handling error as default

0886ae0

chore: move attachment tests to node package

871d3dd

chore: fix comment

3b6a8fe

chore: conform verifyAttachments to normal function

e9ac51a

feat: add abortable attachment watchers

ecd22e5

feat: add vanilla react native storage adapter

8cfc680

fix: web attachment display using blob url

aec02b1

fix: export expo fs adapter

9812964

stevensJourney reviewed Dec 15, 2025

View reviewed changes

joshuabrink added 5 commits December 22, 2025 13:42

refactor: implement watchAttachments as a class property

4946f37

refactor: AttachmentQueue options as immutable config and improve log…

46fac0f

…ger assignment

fix: use private property for timer assignment

65c6384

chore: add adapter deps to rollup external and dev dependencies

5ef0257

fix: add lazy import to constructor and improve error handling

12f533f

stevensJourney mentioned this pull request Dec 25, 2025

Feature Request: Support for modern expo-file-system API (Expo SDK 54+) #803

Open

stevensJourney reviewed Dec 29, 2025

View reviewed changes

joshuabrink added 3 commits December 29, 2025 12:36

chore: update expo file system depedency

03996bb

refactor: enforce exclusive database access via AttachmentService

73a3c06

refactor: improve separation of third party file system dependencies

bd6a5b9

joshuabrink requested a review from stevensJourney January 5, 2026 09:26

stevensJourney reviewed Jan 5, 2026

View reviewed changes

Update attachment package #735

Are you sure you want to change the base?

Update attachment package #735

Uh oh!

Conversation

joshua-journey-apps commented Oct 7, 2025 • edited by khawarizmus Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

stevensJourney left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stevensJourney left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stevensJourney left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

joshua-journey-apps commented Oct 7, 2025 •

edited by khawarizmus

Loading

changeset-bot bot commented Oct 7, 2025 •

edited

Loading