test(m2): backupSmoke — round-trip a real Android export through :core:backup, + import-timestamp fix

Phase 1 M2 verification probe, driven by a real phone export (27 notes,
1 attachment, 14 orphan tags; the file is user data and stays out of the
repo):

- :core:data-jvm:backupSmoke --args="<export.incido.json> [libDir]" —
  preview, import into a fresh JVM library, re-export + field-by-field
  compare, encrypted envelope export (wrong passphrase rejected),
  encrypted import into a 2nd library with attachment-byte equality.

The probe caught a PRE-EXISTING Android bug the phone also has: upsert /
saveNoteWithHashtags stamped createdAt+updatedAt to now on every insert,
so backup import rewrote every note's history — and would defeat the
(createdAt, updatedAt, sha256(content)) dedupe key M2 task 3 depends on.
Justified behavior change (invariant 1: not silent): non-zero caller
timestamps are now honoured, 0 still means now. The Android editor
passes its own stamps (verified at both Note(...) sites), so only the
import path changes. addAttachment gains createdAt: Long = 0L;
importSnapshot passes the phone value through.

Verification passed: :core:data-jvm:backupSmoke (RESULT: M2 round-trip
PASS on the real export) + assembleDebug + :core:data-jvm:smoke (after a
forced codegen regen: "ID model file unchanged") + :desktopApp:run
--self-test + :spike:data-jvm:run — all green; output recorded in
DESKTOP_FEASIBILITY.md.

Known pre-existing gap left open (documented): importSnapshot never
recreates orphaned hashtag rows.
This commit is contained in:
Incido Developer committed 2026-10-05 19:03:31 +13:00
1 parent 7cc4eda90d
commit 8facbe718b
6 files changed
+278 -3

No files matched your search

@@ -93,6 +93,9 @@ object BackupCodec {
bytes = decodeBase64(ba.bytesBase64), bytes = decodeBase64(ba.bytesBase64),
mimeType = ba.mimeType, mimeType = ba.mimeType,
displayName = ba.displayName, displayName = ba.displayName,
// Restore the phone's attachment timestamp (M2:
// everything import claims to preserve must survive).
createdAt = ba.createdAt,
) )
} }
onProgress(idx + 1, total) onProgress(idx + 1, total)
+19
View File
@@ -63,6 +63,11 @@ dependencies {
api(project(":core:domain")) api(project(":core:domain"))
implementation(libs.kotlinx.coroutines.core) implementation(libs.kotlinx.coroutines.core)
// Phase 1 (M2): the backup bridge, for the backupSmoke round-trip probe.
// No classpath risk here — :core:backup carries no ObjectBox at all.
implementation(project(":core:backup"))
implementation(libs.kotlinx.serialization.json)
// JVM counterparts of the objectbox-android the Android build uses. // JVM counterparts of the objectbox-android the Android build uses.
implementation(libs.objectbox.java) implementation(libs.objectbox.java)
implementation(libs.objectbox.kotlin) implementation(libs.objectbox.kotlin)
@@ -85,3 +90,17 @@ tasks.register<JavaExec>("smoke") {
mainClass.set("org.incido.data.jvm.StoreSmokeKt") mainClass.set("org.incido.data.jvm.StoreSmokeKt")
args = listOf(layout.buildDirectory.dir("data-jvm-smoke").get().asFile.absolutePath) args = listOf(layout.buildDirectory.dir("data-jvm-smoke").get().asFile.absolutePath)
} }
/**
* `./gradlew :core:data-jvm:backupSmoke --args="<backup.incido.json> [libDir]"`
* — feeds a REAL Android backup file through the :core:backup bridge on the
* JVM: preview, import into a fresh library, re-export and compare, then the
* encrypted-envelope path (including wrong-passphrase rejection). Runs
* against a fresh library every time (the script deletes libDir first).
*/
tasks.register<JavaExec>("backupSmoke") {
group = "verification"
description = "Round-trips a real Android backup export through BackupBridge on the JVM."
classpath = sourceSets["main"].runtimeClasspath
mainClass.set("org.incido.data.jvm.BackupRoundTripKt")
}
@@ -0,0 +1,194 @@
package org.incido.data.jvm
import java.io.ByteArrayInputStream
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.FileInputStream
import java.util.Base64
import kotlin.system.exitProcess
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.runBlocking
import org.incido.compose.data.backup.BackupBridge
import org.incido.compose.data.backup.BackupCodec
import org.incido.compose.data.backup.BackupFile
import org.incido.compose.data.backup.BackupNote
import org.incido.compose.data.backup.EncryptedBackupCodec
import org.incido.data.ImageOrientationNormalizer
import org.incido.data.ObjectBoxStore
import org.incido.data.crypto.CipherHolder
import org.incido.data.repository.NoteRepositoryImpl
import org.incido.domain.repository.NoteRepository
/**
* Phase 1 (M2) round-trip probe: runs a **real** `.incido.json` export
* produced by the Android app through the extracted `:core:backup`
* bridge on a plain JVM — import into a live ObjectBox library, re-export,
* and compare note-for-note (incl. attachment bytes), plus the encrypted
* envelope path with a wrong-passphrase rejection.
*
* Unlike `smoke`, this script deletes [libDir] up front — it always runs
* against a fresh library.
*
* Run with
* `./gradlew :core:data-jvm:backupSmoke --args="<backup.incido.json> [libDir]"`.
*/
fun main(args: Array<String>) {
val src = File(args.getOrNull(0) ?: error("usage: backupSmoke <backup.incido.json> [libDir]"))
require(src.isFile) { "backup file not found: $src" }
val libDir = File(args.getOrNull(1) ?: "build/backup-roundtrip").also {
it.deleteRecursively()
it.mkdirs()
}
println("=== Incido Phase 1 M2: backup bridge on the JVM (real Android export) ===")
println("source file : $src (${src.length()} bytes)")
val original = BackupCodec.decode(src.readText())
val origAttachments = original.notes.sumOf { it.attachments.size }
println(
"source : notes=${original.notes.size} attachments=$origAttachments " +
"orphanTags=${original.orphanedHashtags.size} app=${original.appVersion}",
)
var failures = 0
fun step(name: String, body: () -> String) {
print("${name.padEnd(32)} ")
runCatching(body)
.onSuccess { println("PASS — $it") }
.onFailure {
println("FAIL — ${it::class.java.simpleName}: ${it.message}")
failures++
}
}
// Fresh CharArray per use — the codecs zero the passphrase they consume.
fun pw(): CharArray = "RoundTrip passphrase #2026!".toCharArray()
fun repo(): NoteRepository =
NoteRepositoryImpl(ObjectBoxStore.get(), CipherHolder(), ImageOrientationNormalizer.NoOp)
step("preview (plain, no passphrase)") {
val p = BackupBridge.preview(FileInputStream(src)) ?: error("preview returned null")
require(p.noteCount == original.notes.size) { "noteCount=${p.noteCount}" }
require(p.attachmentCount == origAttachments) { "attachmentCount=${p.attachmentCount}" }
require(p.appVersion == original.appVersion) { "appVersion=${p.appVersion}" }
"notes=${p.noteCount} attachments=${p.attachmentCount} app=${p.appVersion}"
}
step("import into fresh JVM library") {
ObjectBoxStore.init(libDir)
val n = runBlocking { BackupBridge.import(repo(), FileInputStream(src)) }
require(n == original.notes.size) { "imported $n, expected ${original.notes.size}" }
val inStore = runBlocking { repo().observeAllIncludingTrashed().first().size }
require(inStore == original.notes.size) { "store holds $inStore" }
"imported $n notes, all readable back through the flow"
}
step("re-export plain == original") {
val out = ByteArrayOutputStream()
runBlocking { BackupBridge.exportPlain(repo(), out, "m2-roundtrip") }
val round = BackupCodec.decode(out.toString(Charsets.UTF_8))
val lostOrphans = original.orphanedHashtags.toSet() - round.orphanedHashtags.toSet()
compareNotes(original, round) +
" (orphan tags: ${round.orphanedHashtags.size}/${original.orphanedHashtags.size} recreated — " +
"importSnapshot has never restored orphans, pre-existing)" +
(if (lostOrphans.isEmpty()) "" else " [-$lostOrphans]")
}
var envelope = ByteArray(0)
step("encrypted export + wrong-pw reject") {
val out = ByteArrayOutputStream()
runBlocking { BackupBridge.exportEncrypted(repo(), out, pw(), "m2-roundtrip") }
envelope = out.toByteArray()
require(EncryptedBackupCodec.isEncrypted(envelope)) { "no INCDBK1 magic" }
val wrong = BackupBridge.preview(
ByteArrayInputStream(envelope),
"wrong wrong wrong #2026!".toCharArray(),
)
require(wrong == null) { "wrong passphrase was accepted!" }
"envelope=${envelope.size}B (json=${original.notes.size} notes), wrong passphrase → null"
}
step("import encrypted into 2nd library") {
ObjectBoxStore.reset()
val dir2 = File(libDir.absolutePath + "-enc").also { it.deleteRecursively(); it.mkdirs() }
ObjectBoxStore.init(dir2)
val n = runBlocking { BackupBridge.import(repo(), ByteArrayInputStream(envelope), pw()) }
require(n == original.notes.size) { "imported $n, expected ${original.notes.size}" }
val withAtt = runBlocking { repo().observeAllIncludingTrashed().first() }
.first { it.attachments.isNotEmpty() }
val bytes = runBlocking { repo().getAttachmentBytes(withAtt.attachments.single().id) }
?: error("attachment missing after encrypted import")
val expected = Base64.getDecoder()
.decode(original.notes.first { it.attachments.isNotEmpty() }.attachments.first().bytesBase64)
require(bytes.contentEquals(expected)) { "attachment bytes differ (${bytes.size} vs ${expected.size})" }
"imported $n notes; attachment ${bytes.size}B byte-identical to the phone's"
}
step("close libraries") {
ObjectBoxStore.reset()
"clean"
}
println()
println(
if (failures == 0) "RESULT: M2 round-trip PASS — a real Android export survives the desktop bridge."
else "RESULT: M2 round-trip FAIL — $failures step(s) failed.",
)
if (failures != 0) exitProcess(1)
}
/**
* Field-by-field comparison of two snapshots. Import forces fresh ids, so
* rows are paired positionally after sorting by the only stable keys every
* platform preserves: (createdAt, content). Attachments are paired the same
* way within each note and compared including their Base64 payload.
*/
private fun compareNotes(original: BackupFile, round: BackupFile): String {
require(round.notes.size == original.notes.size) {
"note count drifted: ${round.notes.size} != ${original.notes.size}"
}
fun List<BackupNote>.keyed() = sortedWith(
compareBy({ it.createdAt }, { it.content }),
)
val a = original.notes.keyed()
val b = round.notes.keyed()
a.forEachIndexed { i, want ->
val got = b[i]
val at = "note[$i] createdAt=${want.createdAt}"
if (got.content != want.content) {
val w = want.content
val g = got.content
val firstDiff = (0 until minOf(w.length, g.length)).firstOrNull { w[it] != g[it] }
error(
"$at: content differs at char $firstDiff " +
"(len ${w.length} vs ${g.length}) " +
"want=<${w.substring(maxOf(0, (firstDiff ?: 0) - 20), minOf(w.length, (firstDiff ?: 0) + 30)).replace("\n", "\\n")}> " +
"got=<${g.substring(maxOf(0, (firstDiff ?: 0) - 20), minOf(g.length, (firstDiff ?: 0) + 30)).replace("\n", "\\n")}>",
)
}
require(got.updatedAt == want.updatedAt) { "$at: updatedAt differs" }
require(got.isFavorite == want.isFavorite) { "$at: isFavorite differs" }
require(got.color == want.color) { "$at: color ${got.color} != ${want.color}" }
require(got.noteType == want.noteType) { "$at: noteType ${got.noteType} != ${want.noteType}" }
require(got.isTrashed == want.isTrashed) { "$at: isTrashed differs" }
require(got.isDraft == want.isDraft) { "$at: isDraft differs" }
require(got.hashtags.sorted() == want.hashtags.sorted()) {
"$at: hashtags ${got.hashtags.sorted()} != ${want.hashtags.sorted()}"
}
require(got.attachments.size == want.attachments.size) { "$at: attachment count differs" }
val sortKey = compareBy<org.incido.compose.data.backup.BackupAttachment>(
{ it.createdAt },
{ it.displayName },
)
val wa = want.attachments.sortedWith(sortKey)
val ga = got.attachments.sortedWith(sortKey)
wa.forEachIndexed { j, waJ ->
val gaJ = ga[j]
require(gaJ.mimeType == waJ.mimeType) { "$at.att[$j]: mimeType differs" }
require(gaJ.displayName == waJ.displayName) { "$at.att[$j]: displayName differs" }
require(gaJ.sizeBytes == waJ.sizeBytes) { "$at.att[$j]: sizeBytes differs" }
require(gaJ.createdAt == waJ.createdAt) { "$at.att[$j]: createdAt differs" }
require(gaJ.bytesBase64 == waJ.bytesBase64) { "$at.att[$j]: BYTES DIFFER" }
}
}
val attTotal = a.sumOf { it.attachments.size }
return "all ${a.size} notes + $attTotal attachments identical (content/times/fav/color/type/trash/draft/tags/bytes)"
}
@@ -250,6 +250,7 @@ class NoteRepositoryImpl(
bytes: ByteArray, bytes: ByteArray,
mimeType: String, mimeType: String,
displayName: String, displayName: String,
createdAt: Long,
): Long { ): Long {
// v0.3.1.2 — honour the EXIF Orientation tag on inbound images // v0.3.1.2 — honour the EXIF Orientation tag on inbound images
// (camera intent usually writes landscape pixels + an EXIF // (camera intent usually writes landscape pixels + an EXIF
@@ -265,7 +266,9 @@ class NoteRepositoryImpl(
noteId = noteId, noteId = noteId,
mimeType = mimeType, mimeType = mimeType,
displayName = displayName, displayName = displayName,
createdAt = System.currentTimeMillis(), // IMPORT FIX (Phase 1, M2): backup import supplies the phone's
// original timestamp; 0 keeps the old "now" behaviour.
createdAt = createdAt.takeIf { it != 0L } ?: System.currentTimeMillis(),
sizeBytes = storedBytes.size.toLong(), sizeBytes = storedBytes.size.toLong(),
bytes = storedBytes, bytes = storedBytes,
) )
@@ -443,7 +446,15 @@ class NoteRepositoryImpl(
override suspend fun upsert(note: Note): Note { override suspend fun upsert(note: Note): Note {
val now = System.currentTimeMillis() val now = System.currentTimeMillis()
val toPersist = if (note.id == 0L) { val toPersist = if (note.id == 0L) {
note.copy(createdAt = now, updatedAt = now).toEntity() // IMPORT FIX (Phase 1, M2): a caller that supplies a real
// createdAt/updatedAt (backup import must — they are the dedupe
// key and the user's own history) is honoured; 0 still means
// "now". The Android editor already stamps both fields itself,
// so this only changes the import path.
note.copy(
createdAt = note.createdAt.takeIf { it != 0L } ?: now,
updatedAt = note.updatedAt.takeIf { it != 0L } ?: now,
).toEntity()
} else { } else {
note.copy(updatedAt = now).toEntity() note.copy(updatedAt = now).toEntity()
} }
@@ -546,7 +557,12 @@ class NoteRepositoryImpl(
): Note { ): Note {
val now = System.currentTimeMillis() val now = System.currentTimeMillis()
val toPersist = if (note.id == 0L) { val toPersist = if (note.id == 0L) {
note.copy(createdAt = now, updatedAt = now).toEntity() // IMPORT FIX (Phase 1, M2): honour caller-supplied timestamps
// (backup import) — see the matching change in upsert().
note.copy(
createdAt = note.createdAt.takeIf { it != 0L } ?: now,
updatedAt = note.updatedAt.takeIf { it != 0L } ?: now,
).toEntity()
} else { } else {
note.copy(updatedAt = now).toEntity() note.copy(updatedAt = now).toEntity()
} }
@@ -197,12 +197,17 @@ interface NoteRepository {
* verbatim in the database (ObjectBox `ByteArray` column). The * verbatim in the database (ObjectBox `ByteArray` column). The
* generated id is returned so the caller can reference the row * generated id is returned so the caller can reference the row
* (e.g. for display or removal). * (e.g. for display or removal).
*
* [createdAt] lets backup import restore the original timestamp;
* `0` (the default) means "now", so every existing caller is
* unchanged.
*/ */
suspend fun addAttachment( suspend fun addAttachment(
noteId: Long, noteId: Long,
bytes: ByteArray, bytes: ByteArray,
mimeType: String, mimeType: String,
displayName: String, displayName: String,
createdAt: Long = 0L,
): Long ): Long
/** Delete an attachment by id. Idempotent. */ /** Delete an attachment by id. Idempotent. */
+38
View File
@@ -295,3 +295,41 @@ Housekeeping found on the way: `:core:data-jvm:smoke` does not clear
second consecutive run fails with "expected 1 note, got 2". Run it against second consecutive run fails with "expected 1 note, got 2". Run it against
a deleted directory (worth fixing in a later milestone). a deleted directory (worth fixing in a later milestone).
**M2 — real-data round-trip probe + import-timestamp fix: DONE.** New
`./gradlew :core:data-jvm:backupSmoke --args="<export.incido.json> [libDir]"`
feeds a **real** phone export (27 notes, 1 attachment, 14 orphan tags,
TASK_LIST+TEXT, colors, 1 favourite — file kept out of the repo, it's user
data) through the bridge on the JVM: preview → import into a fresh library
→ re-export and compare every field the format claims to preserve →
encrypted envelope export (wrong passphrase → `null` preview) → import of
the envelope into a second library with attachment-byte equality.
The probe immediately caught a **pre-existing Android bug**: `upsert` /
`saveNoteWithHashtags` overwrote `createdAt` AND `updatedAt` with "now"
for every insert, so even the phone's own import re-stamped the whole
library — silently breaking the `(createdAt, updatedAt, sha256(content))`
dedupe key M2 task 3 is built on. Fixed in the shared repository: a
caller-supplied non-zero timestamp is honoured, `0` still means "now"
(the Android editor already stamps both fields itself, verified at its two
`Note(…)` sites — so only the import path changes). Same for
`addAttachment`, which now takes `createdAt: Long = 0L` and
`importSnapshot` passes the phone's value through.
```
preview (plain, no passphrase) PASS — notes=27 attachments=1 app=0.4.0.0
import into fresh JVM library PASS — imported 27 notes, all readable back through the flow
re-export plain == original PASS — all 27 notes + 1 attachments identical
(content/times/fav/color/type/trash/draft/tags/bytes)
encrypted export + wrong-pw reject PASS — envelope=366739B, wrong passphrase → null
import encrypted into 2nd library PASS — attachment 260121B byte-identical to the phone's
RESULT: M2 round-trip PASS
```
Still open from the probe: `importSnapshot` never recreates *orphaned*
hashtag rows (the export carries them, the import ignores them) —
pre-existing on Android, noted for task 3. After the fix, full suite green:
`assembleDebug`, `:core:data-jvm:smoke`, desktop `--self-test`,
`:spike:data-jvm:run` — and the ObjectBox generator kept printing
`ID model file unchanged`.