Where to get Roots now, once SVN gone?

[barracuda156]

Looks like Darwinbuild does not actually build everything from source, at least by default, and wants some pre-compiled stuff from Roots. The problem is that they are gone, and archive.org does not seem to have preserved anything. For example: page is indexed, but files are unavailable: https://web.archive.org/web/20140223045546/http://src.macosforge.org/Roots/10J869/

When I try to run a build locally, it just fails:

bash-3.2# sudo darwinbuild -headers  xnu
*** Fetching Sources ...
Found xnu-1504.15.3.tar.gz in /Users/svacchanda/10K540/Sources
*** Copying Sources ...
*** Installing Roots ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10J869//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10J869//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10J567//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10H574//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10F569//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10D573//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10C540//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10B504//files.root.tar.gz ...
Downloading http://macos-powerpc.org/darwinbuild/Roots/10A432//files.root.tar.gz ...
ERROR: could not find root: files

(The URL is different on purpose, since I moved Projects there, however I do not know where to borrow Roots from.)

[bhuntsman]

You can rebuild some of them yourself using the -nochroot flag to bootstrap, then you can use those to populate a root.
In later iterations, by the time you get up to High Sierra or Mojave or somewhere in there, using -nochroot is going to be your only option.

[barracuda156]

@bhuntsman Thank you, that seems to work, but now it complains re target:

bash-3.2# sudo darwinbuild -nochroot xnu
*** Fetching Sources ...
Found xnu-1504.15.3.tar.gz in /Users/svacchanda/10K540/Sources
*** Copying Sources ...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BUILDING xnu-1504.15.3~1 on Fri Dec 20 08:23:38 CST 2024
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Build configuration:
    Build host:           macmini.private
    Build tool:           make
    Build action:         install
    Build number:         10K540
    Host kernel version:  Darwin Kernel Version 10.8.0: Tue Jun  7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64
    cc version:           gcc version 4.2.1 (Apple Inc. build 5666) (dot 3)
    cctools version:      ools-795~45, GNU assembler version 1.38
    make version:         GNU Make 3.81
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Build parameters:
SRCROOT: /Users/svacchanda/10K540/BuildRoot/SourceCache/xnu/xnu-1504.15.3
OBJROOT: /Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.obj
SYMROOT: /Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.sym
DSTROOT: /Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.root
RC_ProjectName: xnu
RC_ProjectSourceVersion: 1504.15.3
RC_ProjectNameAndSourceVersion: xnu-1504.15.3
RC_ProjectBuildVersion: 1
INSTALLED_PRODUCT_ASIDES: YES
MACOSX_DEPLOYMENT_TARGET: 10.6
NEXT_ROOT: 
RC_ARCHS: ppc i386 x86_64
RC_CFLAGS: -pipe -arch ppc -arch i386 -arch x86_64
RC_JASPER: YES
RC_NONARCH_CFLAGS: -pipe
RC_OS: macos
RC_PRIVATE: /private
RC_RELEASE: SnowLeopard
RC_TARGET_CONFIG: MacOSX
RC_XBS: YES
RC_i386: YES
RC_ppc: YES
RC_x86_64: YES
SEPARATE_STRIP: YES
UNAME_RELEASE: 10.0
UNAME_SYSNAME: Darwin
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Environment variables:
DARWINBUILD_BUILD=10K540
DARWINXREF_DB_FILE=/Users/svacchanda/10K540/.build/xref.db
DARWIN_BUILDROOT=/Users/svacchanda/10K540
DSTROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.root
GROUP=wheel
HOME=/var/root
INSTALLED_PRODUCT_ASIDES=YES
INSTALL_XCODE=NO
LOGNAME=root
MACOSX_DEPLOYMENT_TARGET=10.6
NEXT_ROOT=
OBJROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.obj
PATH=/bin:/sbin:/usr/bin:/usr/sbin/:/usr/local/bin:/usr/local/sbin
RC_ARCHS=ppc i386 x86_64
RC_CFLAGS=-pipe -arch ppc -arch i386 -arch x86_64
RC_JASPER=YES
RC_NONARCH_CFLAGS=-pipe
RC_OS=macos
RC_PRIVATE=/private
RC_ProjectBuildVersion=1
RC_ProjectName=xnu
RC_ProjectNameAndSourceVersion=xnu-1504.15.3
RC_ProjectSourceVersion=1504.15.3
RC_RELEASE=SnowLeopard
RC_TARGET_CONFIG=MacOSX
RC_XBS=YES
RC_i386=YES
RC_ppc=YES
RC_x86_64=YES
SEPARATE_STRIP=YES
SHELL=/bin/sh
SRCROOT=/Users/svacchanda/10K540/BuildRoot/SourceCache/xnu/xnu-1504.15.3
SYMROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.sym
UNAME_RELEASE=10.0
UNAME_SYSNAME=Darwin
USER=root
USE_CHROOT=NO
_=/usr/bin/printenv
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Installed Roots:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
make install  "SRCROOT=/Users/svacchanda/10K540/BuildRoot/SourceCache/xnu/xnu-1504.15.3" "OBJROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.obj" "SYMROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.sym" "DSTROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/xnu/xnu-1504.15.3.root" "RC_ProjectName=xnu" "RC_ProjectSourceVersion=1504.15.3" "RC_ProjectNameAndSourceVersion=xnu-1504.15.3" "RC_ProjectBuildVersion=1" "INSTALLED_PRODUCT_ASIDES=YES" "MACOSX_DEPLOYMENT_TARGET=10.6" "NEXT_ROOT=" "RC_ARCHS=ppc i386 x86_64" "RC_CFLAGS=-pipe -arch ppc -arch i386 -arch x86_64" "RC_JASPER=YES" "RC_NONARCH_CFLAGS=-pipe" "RC_OS=macos" "RC_PRIVATE=/private" "RC_RELEASE=SnowLeopard" "RC_TARGET_CONFIG=MacOSX" "RC_XBS=YES" "RC_i386=YES" "RC_ppc=YES" "RC_x86_64=YES" "SEPARATE_STRIP=YES" "UNAME_RELEASE=10.0" "UNAME_SYSNAME=Darwin" < /dev/null
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Build log begins here:

make: *** No rule to make target `install'.  Stop.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 BUILD TIME: 0h 0m 0s
EXIT STATUS: 2

Do I need to pass some extra argument in this case?

(Sorry for a stupid question, I am just unfamiliar with this build system. Once I have it working in one case, I won’t keep needing assistance.)

[bhuntsman]

Something else is missing... I'll fire up my old DarwinBuild host in a bit...
You can view the plist settings with "darwinxref edit darwinxref currentBuild"
Hopefully you have some extra stuff in there and not just the plist info as distributed from Apple...

[bhuntsman]

er, sorry, those are supposed to be backticks around darwinxref currentBuild

[barracuda156]

The only functional modification I made to plists, as of now, was to add ppc arch (since that is what I actually need, whether x86 is used or not). 10K540 plist does not even set arch, so it is indeed “as is” (how it exists in Darwinbuild at least).

[ryandesign]

The problem is that they are gone, and archive.org does not seem to have preserved anything

I still have these files from when I decommissioned the macOS forge servers in 2016.

Our goal at the time was to preserve everything that existed and redirect old URLs to new ones.

It looks like at the time, everything under http://src.macosforge.org/ was configured to redirect to equivalent URLs under https://opensource.apple.com/darwinbuild/ ; see for example:

https://web.archive.org/web/20201025062500/https://src.macosforge.org/Roots/

In the years since then, it appears that Apple has removed https://opensource.apple.com/darwinbuild/ and changed the src.macosforge.org redirects to point to the darwinbuild homepage. I don't know whether Apple preserved the old darwinbuild sources elsewhere or deleted them. If they deleted them and you need them I could make them available on my server.

[bhuntsman]

Oh man, anything you could publish would be extremely interesting!

Also, do you know who currently has admin rights on this repo? It would be interesting if we could get Apple to turn it over to the community.

[ryandesign]

everything under http://src.macosforge.org/ was configured to redirect to equivalent URLs under https://opensource.apple.com/darwinbuild/

Specifically, this was the Apache conf file I wrote:

<VirtualHost *>
  ServerName src.macosforge.org

  RewriteEngine On
  RewriteRule ^/+$ https://macosforge.github.io/darwinbuild/ [L,R=permanent]
  RewriteRule ^/+Projects/+((.+)-.+\.tar\.gz)$ https://opensource.apple.com/tarballs/$2/$1 [L,R=permanent]
  RewriteRule ^(.*)$ https://opensource.apple.com/darwinbuild$1 [L,R=permanent]
</VirtualHost>

[ryandesign]

@barracuda156 I don't have the bandwidth to host these files publicly; do you? If so you could rsync them from me. It's about 8.2 GiB.

[barracuda156]

@ryandesign Everything for 10.5.x and 10.6.x are of interest and immediate relevance, and these we can host publicly, I believe. How do I rsync from you? I mean, where exactly from?

[barracuda156]

@bhuntsman It is very much likely that something is just missing in my set-up, since I get the same error with cctools for example:

bash-3.2# darwinbuild -nochroot cctools
*** Fetching Sources ...
Downloading http://macos-powerpc.org/darwinbuild/Projects//cctools-750.tar.gz ...
Download complete
*** Copying Sources ...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
BUILDING cctools-750~1 on Fri Dec 20 10:27:29 CST 2024
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Build configuration:
    Build host:           macmini.private
    Build tool:           make
    Build action:         install
    Build number:         10K540
    Host kernel version:  Darwin Kernel Version 10.8.0: Tue Jun  7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64
    cc version:           gcc version 4.2.1 (Apple Inc. build 5666) (dot 3)
    cctools version:      ools-795~45, GNU assembler version 1.38
    make version:         GNU Make 3.81
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Build parameters:
SRCROOT: /Users/svacchanda/10K540/BuildRoot/SourceCache/cctools/cctools-750
OBJROOT: /Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.obj
SYMROOT: /Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.sym
DSTROOT: /Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.root
RC_ProjectName: cctools
RC_ProjectSourceVersion: 750
RC_ProjectNameAndSourceVersion: cctools-750
RC_ProjectBuildVersion: 1
INSTALLED_PRODUCT_ASIDES: YES
MACOSX_DEPLOYMENT_TARGET: 10.6
NEXT_ROOT: 
RC_ARCHS: ppc i386 x86_64
RC_CFLAGS: -pipe -arch ppc -arch i386 -arch x86_64
RC_JASPER: YES
RC_NONARCH_CFLAGS: -pipe
RC_OS: macos
RC_PRIVATE: /private
RC_RELEASE: SnowLeopard
RC_TARGET_CONFIG: MacOSX
RC_XBS: YES
RC_i386: YES
RC_ppc: YES
RC_x86_64: YES
SEPARATE_STRIP: YES
UNAME_RELEASE: 10.0
UNAME_SYSNAME: Darwin
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Environment variables:
DARWINBUILD_BUILD=10K540
DARWINXREF_DB_FILE=/Users/svacchanda/10K540/.build/xref.db
DARWIN_BUILDROOT=/Users/svacchanda/10K540
DSTROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.root
GROUP=wheel
HOME=/var/root
INSTALLED_PRODUCT_ASIDES=YES
INSTALL_XCODE=NO
LOGNAME=root
MACOSX_DEPLOYMENT_TARGET=10.6
NEXT_ROOT=
OBJROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.obj
PATH=/bin:/sbin:/usr/bin:/usr/sbin/:/usr/local/bin:/usr/local/sbin
RC_ARCHS=ppc i386 x86_64
RC_CFLAGS=-pipe -arch ppc -arch i386 -arch x86_64
RC_JASPER=YES
RC_NONARCH_CFLAGS=-pipe
RC_OS=macos
RC_PRIVATE=/private
RC_ProjectBuildVersion=1
RC_ProjectName=cctools
RC_ProjectNameAndSourceVersion=cctools-750
RC_ProjectSourceVersion=750
RC_RELEASE=SnowLeopard
RC_TARGET_CONFIG=MacOSX
RC_XBS=YES
RC_i386=YES
RC_ppc=YES
RC_x86_64=YES
SEPARATE_STRIP=YES
SHELL=/bin/sh
SRCROOT=/Users/svacchanda/10K540/BuildRoot/SourceCache/cctools/cctools-750
SYMROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.sym
UNAME_RELEASE=10.0
UNAME_SYSNAME=Darwin
USER=root
USE_CHROOT=NO
_=/usr/bin/printenv
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Installed Roots:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
make install  "SRCROOT=/Users/svacchanda/10K540/BuildRoot/SourceCache/cctools/cctools-750" "OBJROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.obj" "SYMROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.sym" "DSTROOT=/Users/svacchanda/10K540/BuildRoot/private/var/tmp/cctools/cctools-750.root" "RC_ProjectName=cctools" "RC_ProjectSourceVersion=750" "RC_ProjectNameAndSourceVersion=cctools-750" "RC_ProjectBuildVersion=1" "INSTALLED_PRODUCT_ASIDES=YES" "MACOSX_DEPLOYMENT_TARGET=10.6" "NEXT_ROOT=" "RC_ARCHS=ppc i386 x86_64" "RC_CFLAGS=-pipe -arch ppc -arch i386 -arch x86_64" "RC_JASPER=YES" "RC_NONARCH_CFLAGS=-pipe" "RC_OS=macos" "RC_PRIVATE=/private" "RC_RELEASE=SnowLeopard" "RC_TARGET_CONFIG=MacOSX" "RC_XBS=YES" "RC_i386=YES" "RC_ppc=YES" "RC_x86_64=YES" "SEPARATE_STRIP=YES" "UNAME_RELEASE=10.0" "UNAME_SYSNAME=Darwin" < /dev/null
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Build log begins here:

make: *** No rule to make target `install'.  Stop.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 BUILD TIME: 0h 0m 0s
EXIT STATUS: 2

On a side note, contrary to documentation, export ARCHS="ppc i386" seems to be ignored; apparently, build archs are picked from plists, discarding env settings and archs of Darwinbuild itself.

[bhuntsman]

"normally", your build should be something like "10K540od1" to differentiate it from the stock plists. Then, if you do "darwinxref edit 10K540od1", you should have a section like this:

    environment = {
            INSTALLED_PRODUCT_ASIDES = YES;
            MACOSX_DEPLOYMENT_TARGET = 10.6;
            NEXT_ROOT = "";
            RC_ARCHS = "ppc i386";
            RC_JASPER = YES;
            RC_NONARCH_CFLAGS = "-pipe -no-cpp-precomp";
            RC_OS = macos;
            RC_PRIVATE = /private;
            RC_RELEASE = SnowLeopard;
            RC_XBS = YES;
            SEPARATE_STRIP = YES;
            UNAME_RELEASE = 10.0;
            UNAME_SYSNAME = Darwin;
    };

Under the projects key, you should have a modification for xnu:

xnu = {
environment = {
RC_ARCHS = "i386 ppc";
};

Beyond that, I don't have anything else that jumps out as odd to me...

[bhuntsman]

And yes, the RC_ARCHS gets set by the build system regardless of what you export. You can control it by editing the project using darwinxref.

[bhuntsman]

Just reviewing some of my decade old build logs, and indeed they run make install. Looks like it's just not finding the makefile. IIRC, in the early days, up through 10.7 or .8, you had to do the build on a UFS filesystem because HFS+ is case-preserving but case insensitive. You might try setting up a sparse disk image and dropping a UFS filesystem on it and trying your build from there...

[ryandesign]

How do I rsync from you? I mean, where exactly from?

I've sent you the address by email.