Requirements for anyone who wants to introduce external plugin support for NanaZip

[MouriNaruto]

I also have received some suggestions about introducing the format plugin support for NanaZip. But I need to consider a lot before that because I'm the author of https://github.com/M2TeamArchived/NSudo which made me learn a lot and I know the security isolation is important for plugin host. I want to reduce the attacking surface to reduce the abuse via malicious things. Also, portability is important, which makes me try to pay attention to the WebAssembly with the customized interfaces provided by the plugin host.

According to the words, I mentioned in https://github.com/M2Team/NanaZip/releases/tag/5.0.1250.0.

Currently, NanaZip doesn't accept any plugin-related PRs because many design issues need to be considered first unless you can prove your design is OK for security and portability.

Also, the portability means the binary portability because I think the binary portability of the plugin is important. For example, if the developer builds a plugin, there is only one CPU target version in most cases. People who need plugins will meet the issue when they change the CPU like from x64 to ARM64. We need a solution for solving the issue to bring users a universal experience.

But we always welcome to merge builtin codec PRs if it's decoupled from 7-Zip mainline's source code definitions a.k.a. uses https://github.com/M2Team/NanaZip/blob/main/NanaZip.Specification/NanaZip.Specification.SevenZip.h.

Kenji Mouri

[modz2014]

but all i did was change the location of where to load these plugins I'm confused

[MouriNaruto]

but all i did was change the location of where to load these plugins I'm confused

It's not OK for security. We need a robust isolation model.

It's also not OK for binary portability.

Kenji Mouri

[MouriNaruto]

I don't want to provide external format plugin support unless we can achieve binary portability. (For example, x86, x64 and arm64 all can use the same plugin binary.)

Kenji Mouri

[modz2014]

@MouriNaruto where do you suggest the location of these plugins then we are talking about when the user installs the msix package right if you don't want to change the location then NanaZip will not support plugins WindowsApps folder is read only

[MouriNaruto]

@MouriNaruto where do you suggest the location of these plugins then we are talking about when the user installs the msix package right if you don't want to change the location then NanaZip will not support plugins WindowsApps folder is read only

It's not solving the problem that I mentioned. (binary-level portability and security)

Actually, I hope all external plugins running in the byte code virtual machines which byte code is not associated with any actual CPU ISAs. Also, based on that we should provide limited APIs for plugins for reduce the attack surface.

Kenji Mouri

[modz2014]

@MouriNaruto one question about the msi parse can you point in the right direction to make sure this parses the files correctly atm it does not do it correctly

[MouriNaruto]

@MouriNaruto one question about the msi parse can you point in the right direction to make sure this parses the files correctly atm it does not do it correctly

I don't want to make msi version, but portable version will be done in Classic. (Because support WIndows 11 context menu need AppXManifest, and I'm not good at handling the Sparse Package mode.)

Kenji Mouri

[modz2014]

When opening an MSI file in NanoZip or other archiving tools, it might not display the contents properly

[MouriNaruto]

When opening an MSI file in NanoZip or other archiving tools, it might not display the contents properly

for msi format parsing bugs should find igor because it's inherited from 7-Zip mainline

Kenji Mouri

[modz2014]

@MouriNaruto already tried contacting him he doesn't care that's why I moved form from him

[modz2014]

@MouriNaruto Also the progress bar colour should be blue since we are on dark mode like a blue

[dinhngtu]

Actually, I hope all external plugins running in the byte code virtual machines which byte code is not associated with any actual CPU ISAs. Also, based on that we should provide limited APIs for plugins for reduce the attack surface.

I'm worried about the performance cost and ABI mismatch between the bytecode environment (WASM?) and the native libraries. How about making plugins runnable in a separate process, with a secure RPC pipe between them and the main process? You can do the same with the NanaZip.Core interface.

When opening an MSI file in NanoZip or other archiving tools, it might not display the contents properly

You should use Orca or a similar tool for MSI files. MSI files are databases and archive tools just aren't designed to process them.

[MouriNaruto]

@dinhngtu

I'm worried about the performance cost and ABI mismatch between the bytecode environment (WASM?) and the native libraries.

Actually, I don't want to provide the external plugins support with the native interfaces because I cannot make sure plugin developers to make plugins for all platforms supported for NanaZip. I don't want to make something like x86-64 Windows user mode emulator for NanaZip in the future because some popular external plugins relies on that. This is what I concerned.

Of course, I will make a lightweight WASM VM with JIT support for that case, it will reduce some overheads. For ABI, external plugins will use the limited interfaces to implement for portability and security.

Some useful notes for my opinion: NanaZip will have the POSIX edition called AptxZip in the recent years, which prototype development platform is FreeBSD. (In the next NanaZip preview, I will add some new AppX execution aliases like K7.exe, K7C.exe and K7G.exe to NanaZip, one reason is for more simplified command line user experience, the other reason is NanaZip for POSIX a.k.a. AptxZip will use the scheme but only provide /bin/k7c and /bin/k7t, k7t is the TUI edition (of course, with the mouse support). I think TTY and SSH are the native experience for POSIX, like the Win32 GUI subsystem and RDP are the native experience for Windows.) I hope both NanaZip and AptxZip will share the same external plugin ecosystem. (Also, I will be happy if can share more things.)

Kenji Mouri

[modz2014]

@MouriNaruto Got it, thanks for the detailed explanation! A lightweight WASM VM with JIT support sounds like a solid solution for reducing overhead. I'm curious about how the limited interfaces for plugins will work across platforms, but the focus on security and portability definitely makes sense. Also, if you don’t plan to support plugins, I assume you'll continue updating NanaZip with new features to keep things evolving.

@dinhngtu I agree that using Orca or a similar tool for MSI files is a better option, as archiving tools aren’t really designed for them. I also like the idea of running plugins in a separate process with secure RPC pipes to minimize performance issues — that could definitely streamline things. As for plugins, I’ve already made four for 7zip, but doing it solo can be pretty annoying without any help. The 7zip developer hasn’t been much assistance, so I’ve done a lot of research on my own. That said, I know others are still learning too.

[MouriNaruto]

@modz2014

I'm curious about how the limited interfaces for plugins will work across platforms

The plugin context will be running in the WASM VM, and will get the same plugin interfaces for all platform targets. (Read WASI will give you some help, although NanaZip's portable plugin host will not support WASI, and use the simple interfaces instead.)

Kenji Mouri

[modz2014]

I'm not worried about portable version

[MouriNaruto]

I'm not worried about portable version

The "portable" I mean is cross arch and platform. Maybe I should use the word "portability".

Kenji Mouri

[modz2014]

yes haha but most plugins wont be cross platform

[MouriNaruto]

@modz2014

yes haha but most plugins wont be cross platform

This is what I concerned. I need that. So, I choose the byte-code VM style solution.

Kenji Mouri

[modz2014]

but there is already 7zip for other platforms with there own plugins

[MouriNaruto]

@modz2014

Referenced from https://github.com/M2Team/NanaZip/releases/tag/3.0.996, we will know:

NanaZip will be not only a simple 7-Zip fork anymore, the relationship between NanaZip and 7-Zip is more like Chromium and WebKit because I think most of the changes I will do for NanaZip have no chance of being accepted by Igor Pavlov. But NanaZip will still keep the Core Library and Self Extracting Executables implementation synchronized to the latest 7-Zip mainline implementation.

So, I can choose to introduce the new external plugin infrastructures, and I never stop other people from adapting that, that's already enough.

Kenji Mouri

[modz2014]

Perhaps we could leave it as it is for the time being and make updates incrementally? Your current setup would necessitate altering my plugins for compatibility with NanaZip.

[MouriNaruto]

@modz2014

Perhaps we could leave it as it is for the time being and make updates incrementally?

According to #563 (comment),

Actually, I don't want to provide the external plugins support with the native interfaces because I cannot make sure plugin developers to make plugins for all platforms supported for NanaZip. I don't want to make something like x86-64 Windows user mode emulator for NanaZip in the future because some popular external plugins relies on that. This is what I concerned.

I don't think I should introduce any future possible ABI break things like Windows Phone 7 to Windows Phone 8 or LoongArch Old World to LoongArch New World. It's bad for the overall ecosystem. So, I will make that (a.k.a. WebAssembly-based plugin infrastructure) done at the beginning of introducing the external plugin support, and won't leave anything like native external plugin support because I don't want to break the ABIs I have promised.

Kenji Mouri

[modz2014]

@MouriNaruto i'm confused now