fix: [WIDGET-WORKS] Fix LID messages not syncing to Chatwoot during historical sync

[wahyuwidgetworks]

Summary

Fixes LID (Linked Identity Device) messages not syncing to Chatwoot during historical sync (syncFullHistory feature). During historical sync, Baileys provides stripped message keys without the remoteJidAlt field, preventing LID→phone resolution that works correctly in real-time message processing.

This fix adds batch LID resolution using the IsOnWhatsapp table before processing messages in the messages.set handler, ensuring historical LID messages are properly imported to Chatwoot.

Problem Statement

Impact

• Priority: P1 - High
• Affected instances: 8 production instances
• Messages affected: 942 unsynced LID messages identified
• User impact: Customer service agents cannot see complete conversation history for ~5-15% of contacts (WhatsApp multi-device users)

Root Cause

File: src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts:1045

During historical sync (messages.set event), Baileys provides stripped message keys containing only {id, fromMe, remoteJid}. Critical fields are undefined:

• ❌ remoteJidAlt - not available (required for LID resolution)
• ❌ senderPn - not available
• ❌ addressingMode - not available

Why real-time works but historical doesn't:

Flow	remoteJidAlt available?	LID resolved?	Result
Real-time (messages.upsert)	✅ YES	✅ YES (line 1521-1522)	Messages import correctly
Historical (messages.set)	❌ NO (stripped keys)	❌ NO	Messages missed

A prior fix attempt (commit 630f5c56, July 2025) tried using senderPn, but this field is not available in stripped keys during historical sync.

Solution

Implementation

Part 1: Batch LID Resolution ✅ Implemented
Added LID→phone resolution in messages.set handler before message processing:

1. Extract all unique @lid remoteJids from the message batch
2. Batch query IsOnWhatsapp table with OR conditions on jidOptions field
3. Build Map<lidJid, phoneJid> for O(1) lookup
4. Replace remoteJid with phone number where mapping exists
5. Allow unresolved LIDs to pass through (create temporary contacts)

Part 2: Self-Healing ✅ Already exists
Existing code at chatwoot.service.ts:644-666 automatically updates LID contacts with real phone numbers or merges contacts when real-time messages arrive.

Feature Flag

• ENV variable: CHATWOOT_LID_HISTORICAL_SYNC_ENABLED
• Default: true (enabled by default, opt-out with =false)
• Rationale: Backfill already completed successfully, proving the approach works

Changes

Files Modified (3 files, 75 insertions)

1. src/api/integrations/channel/whatsapp/whatsapp.baileys.service.ts (+68 lines)

   • Added batch LID resolution before messages.set processing loop (line 1045)
   • Comprehensive verbose logging for monitoring resolution stats
   • Error handling with graceful degradation (continue on failure)
   • Clear [WIDGET-WORKS] labeling with context comments

2. src/config/env.config.ts (+6 lines)

   • Added LID_HISTORICAL_SYNC_ENABLED to Chatwoot type definition
   • Added feature flag implementation with default true
   • Clear [WIDGET-WORKS] comments for upstream merge tracking

3. CHANGELOG.md (+1 line)

   • Added entry to Widget Works Modifications - Tier 1 (Always On)

Testing Plan

Pre-Deployment Verification

• ✅ ESLint passed (no linting errors)
• ✅ TypeScript compilation passed (no type errors)
• ✅ Pre-commit hooks passed (husky + lint-staged)

Post-Deployment Testing

Phase 1: Single Test Instance

1. Deploy to one test instance
2. Enable verbose logging: LOG_BAILEYS=verbose
3. Trigger reconnect to generate messages.set event
4. Monitor logs for:

   [messages.set] Found X unique LID identifiers, attempting resolution via IsOnWhatsapp table
   [messages.set] Resolved Y/X LID identifiers to phone numbers
   [messages.set] Replaced Z message remoteJids from LID to phone
   

5. Verify LID messages now appear in Chatwoot with correct contact names
6. Check self-healing: trigger real-time message from unresolved LID, verify contact updates

Phase 2: Production Rollout
7. If test successful, deploy to remaining 7 instances
8. Monitor for any errors or unexpected behavior
9. Verify historical sync no longer misses LID messages

Rollback Plan

# Disable feature if issues arise
CHATWOOT_LID_HISTORICAL_SYNC_ENABLED=false

Upstream Compatibility

Research Summary

• Upstream version: v2.3.7 already deployed (contains PR Fix: @lid problems, messages events and chatwoot integration errors EvolutionAPI/evolution-api#2275 for real-time LID fixes)
• This fix: Addresses gap in historical sync not covered by upstream
• Merge path: Minimal divergence - single focused feature with clear [WIDGET-WORKS] labeling
• Future merge: Comments provide full context for upstream maintainers

Related Upstream Changes

• PR Fix: @lid problems, messages events and chatwoot integration errors EvolutionAPI/evolution-api#2275: Multiple fixes for @lid problems, message events (real-time)
• PR fix: unify remoteJid filtering using OR with remoteJidAlt EvolutionAPI/evolution-api#2249: Unify remoteJid filtering using OR with remoteJidAlt
• PR Fix Typebot message routing for @lid JIDs EvolutionAPI/evolution-api#2264: Fix Typebot message routing for @lid JIDs
• PR fix: Erro ao enviar mensagem para grupos (remoteJid failed constraint) EvolutionAPI/evolution-api#2162: Fix OnWhatsapp cache with OR jidOptions/remoteJid

Verification

Manual Backfill Results (Completed 2026-01-30)

• ✅ 670 messages imported across 8 instances
• ✅ 0 errors during backfill
• ✅ All unsynced LID messages now in Chatwoot
• ✅ Self-healing verified (62 "Você" contacts fixed)

This fix prevents the issue from recurring on future reconnects.

Additional Context

• Issue tracking: 🐛 syncFullHistory missing messages from LID (multi-device) contacts #17
• Investigation docs: docs/troubleshooting/30-01-26-chatwoot-syncfullhistory-lid-missing.md
• Backfill script: scripts/backfill-lid-to-chatwoot.ts (already executed)

---

Reviewers: Please verify:

• Feature flag implementation matches project patterns
• Logging is appropriate (verbose level, not info/error spam)
• Error handling allows graceful degradation
• Comments provide sufficient context for future maintenance
• TypeScript types are properly updated