I’ve been reading up on how submodules can be bad for a git repository since all they do is store SHA and compare them with it’s parent repository to see if they are changed. If not correctly manage, they could lead to data lose. If that is the case, what are the alternatives for us to use besides submodules? I know MyPaint will be soon divided into three repositories so we should talk about how we should manage them so they don’t break each other.
Hi,
A submodule is basically conceptually a mix between separate repositories and an all in one repository. When you use libmypaint as a submodule, you are considering its code in the same repository and reinstalling it when installing mypaint. This goes against the point of having a separate and independent libmypaint. I want to have a libmypaint installed separately and when I build MyPaint, it should detect this existing libmypaint, its version, its path, its compile and linking flags, and just use it. GIMP as well will share the same libmypaint installation.
This is exactly what pkg-config is for. That’s good because libmypaint already has a .pc file. Just get rid of the submodule, and just detect libmypaint at construction time. That’s what we do in GIMP to use libmypaint.
And it should be the same for any data — not only libraries — which is meant to be shared with other applications, like brushes (Brushes hosted in mypaint-brushes and libmypaint as a normal dependency by Jehan · Pull Request #538 · mypaint/mypaint · GitHub).
The alternative we need is basically installing all those “in-house” dependencies into the same target tree. There are a few wrinkles of course; there are always fiddly consequences:
- In-tree testing (used every day in MyPaint development) will become a bit more confusing.
- The Ubuntu PPA build (our “debian” module, currently) will need to be split (and then we’d express mypaint → libmypaint as a package dependency)
- The Windows build scripting may need to move to a separate module too. At the very least it will need to pull down/build/install libmypaint.dll to its target tree before building MyPaint against it. Fun times.
But I’m quietly confident we’ll be able to get it right. If necessary, MyPaint can do version checking at runtime, or its build system can check the available libmypaint.pc. I guess.
Splitting gives us one important advantage: freedom from SCons! 
I really want to use conventional autotools for libmypaint, and conventional Python distutils for MyPaint one day. For now though, we can depend on SCons still.
I have no idea what “build system” a split-out brushes module should use. I guess we can start with a basic SCons installer though, provided it supports our existing sandbox install oddities.
(I’ll understand more when I have time to attempt this. Hopefully this weekend.)
You usually don’t need to do runtime version checking for dependencies (unless they are runtime dependencies but that’s not the case of libmypaint). pkg-config handles perfectly well build-time version checking.
I think it is a lot less a problem to use a less powerful build system (like SCons) for data only. Well unless part of the data is “built”, in which case I would still go a build system which has at least a support for VPATH builds (to separate sources from built data).
Other than this, you obviously have less concepts to take care of in data-only packages (like cross-compilation is usually meaningless here, unless you have data specifics to a platform).
So yes, SCons won’t be a problem there (even though autotools definitely would not hurt), I think.
