Skip to content

add initial native impl#117

Open
stelzo wants to merge 5 commits intomainfrom
native-unpack
Open

add initial native impl#117
stelzo wants to merge 5 commits intomainfrom
native-unpack

Conversation

@stelzo
Copy link
Copy Markdown
Member

@stelzo stelzo commented Apr 12, 2026

fixes #114

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an initial native (non-Docker) implementation for Dofus 3 Unity bundle/image/i18n unpacking to avoid Docker Desktop UID/GID issues on Windows (fixes #114), while keeping a Docker backend available via DODUDA_UNITY_BACKEND.

Changes:

  • Introduce Unity unpack backend abstraction with native as default and docker as opt-in.
  • Implement native Unity bundle decoding, imagebundle extraction, and i18n .bin parsing (plus a native i18n test).
  • Add a mapping-time normalization step to rewrite rid fields to strings for Unity JSON inputs.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
update.go Switch Unity bundle/image unpacking to backend selection; image pulls now via backend Prepare.
unity_backend.go Adds backend selection via DODUDA_UNITY_BACKEND (default native).
unity_backend_native.go Native backend adapter wiring into new native unpackers.
unity_backend_docker.go Docker backend moved here (container-based bundle/image unpack + image pulling).
unity_bundle_native.go Native UnityFS bundle reader + type tree decoding into JSON.
unity_imagebundle_native.go Native .imagebundle sprite/texture extraction and PNG writing.
unity_i18n_native.go Native localization .bin to JSON converter.
unity_i18n_native_test.go Test covering native i18n conversion using an on-repo fixture.
mapping.go Adds RID normalization pass over Unity raw JSON files prior to parsing.
images.go Adds fallback logic for directory layout differences between backends.
README.md Updates docs to reflect native Dofus 3 unpacking and Docker being optional/legacy.
go.mod / go.sum Adds deps required for native Unity decoding/decompression and texture handling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread unity_backend_docker.go Outdated
Comment thread unity_backend_docker.go
Comment on lines +142 to +146
feedbacks := make(chan string)

var feedbackWg sync.WaitGroup
if !muteSpinner {
feedbackWg.Add(1)
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PullImages creates an unbuffered feedbacks channel and only starts a consumer goroutine when muteSpinner is false. If muteSpinner is true, any send to feedbacks later in the function will block forever. Consider buffering the channel, always consuming it, or guarding sends when muteSpinner is true.

Copilot uses AI. Check for mistakes.
Comment thread unity_backend_docker.go Outdated
Comment thread unity_backend_docker.go
Comment on lines +168 to +171
_, err = io.ReadAll(imageHandle)
if err != nil {
log.Fatalf("Error finishing image handler: %v", err)
}
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PullImages returns an error in its signature, but io.ReadAll errors currently call log.Fatalf which terminates the process and skips caller error handling. Prefer returning the error (wrapped with context) instead of exiting.

Copilot uses AI. Check for mistakes.
Comment thread update.go Outdated
feedbackWg.Wait()

return nil
return unpackUnityI18nNative(inputPath, outputPath)
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UnpackUnityI18n bypasses the selected Unity backend and always uses the native implementation, even when DODUDA_UNITY_BACKEND=docker is set. If users force the Docker backend for consistency/workarounds, this behavior is surprising; consider routing i18n unpacking through the backend selection as well (e.g., add it to UnityUnpackBackend).

Copilot uses AI. Check for mistakes.
Comment thread images.go Outdated
errorChan <- err
}
} else if errors.Is(err, os.ErrNotExist) {
if _, toErr := os.Stat(toPath); toErr != nil {
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fallback branch triggers on any error from os.Stat(toPath), not just "not exists". For example, permission or transient IO errors would incorrectly send the code down the native-backend fallback path. Consider checking errors.Is(toErr, os.ErrNotExist) explicitly (and returning other errors).

Copilot uses AI. Check for mistakes.
Comment thread mapping.go Outdated
Comment on lines +96 to +97
if err := os.WriteFile(filePath, rewritten, os.ModePerm); err != nil {
return fmt.Errorf("write %s: %w", entry.Name(), err)
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

normalizeUnityRIDTypes rewrites the original raw JSON files in-place (and re-serializes them without preserving formatting). This is a potentially surprising/destructive side effect for a "map" operation. Consider normalizing during parsing, writing normalized copies to a temp/work directory, or scoping the rewrite to files known to require RID normalization.

Copilot uses AI. Check for mistakes.
Comment thread unity_imagebundle_native.go Outdated
return append([]byte(nil), out...), nil
}

func unityCropSpriteImage(src image.Image, sprite *uni.Sprite) image.Image {
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unityCropSpriteImage is currently unused. If sprites need to be cropped to TextureRect (which this helper suggests), the current export likely writes full textures instead of per-sprite images. Either wire this into the sprite decode/export path or remove the dead code to avoid confusion.

Copilot uses AI. Check for mistakes.
stelzo and others added 3 commits April 13, 2026 13:31
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@stelzo
Copy link
Copy Markdown
Member Author

stelzo commented Apr 13, 2026

@copilot apply changes based on the comments in this thread

…g, stat check, in-place normalization, dead code

Agent-Logs-Url: https://github.com/dofusdude/doduda/sessions/e8c76188-9498-45ba-a041-9bc8b48959f6

Co-authored-by: stelzo <40026170+stelzo@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 13, 2026

@copilot apply changes based on the comments in this thread

Applied all changes from the review thread in commit 554dbd0:

  • feedbacks channel blocking (docker:145): guarded sends with if !muteSpinner to prevent deadlock when no consumer goroutine runs
  • log.Fatalf → return error (docker:170): replaced with return fmt.Errorf("error finishing image handler: %w", err)
  • UnpackUnityI18n backend routing (update.go:553): added UnpackI18n to the UnityUnpackBackend interface; both native and docker backends implement it (docker delegates to native since there's no Docker container for i18n)
  • os.Stat error check (images.go:407): changed toErr != nil to errors.Is(toErr, os.ErrNotExist) and added an else if toErr != nil branch to propagate non-ErrNotExist errors
  • In-place normalization (mapping.go:96): normalizeUnityRIDTypes now takes a dstDir and writes normalized copies there without touching source files; the caller creates a temp dir, passes it for parsing, and defers cleanup
  • Unused unityCropSpriteImage (unity_imagebundle_native.go:330): removed the function and the now-unused image/draw import

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error: uids and gids must be in range 0-2147483647 on Windows with Docker Desktop 29.2.1

3 participants