Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically detect all cores when the --use-all-cores flag is passed #227

Open
wants to merge 60 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
2112c3e
Fix PY2 error
karolyi Mar 26, 2018
d0d482c
Update test versions
asedeno Jan 14, 2019
58486b5
Update test expected queries
asedeno Jan 14, 2019
217ae5d
test setting tweaks for django 2.1
asedeno Jan 14, 2019
9b691a4
user friendly modifications to install_xapian.sh...
kneufeld Nov 4, 2017
03f7a3a
Build fix
memegauste Jun 26, 2019
8f3e129
Fix package, because why not
memegauste Aug 12, 2020
4475d45
Update manual plz
memegauste Jun 14, 2021
9963e54
Move rst to md code
memegauste Jun 14, 2021
51efad6
Fix formatting
memegauste Jun 14, 2021
42fe244
I found it :)
memegauste Jun 14, 2021
236a815
Yh
memegauste Jun 14, 2021
eef148f
Ehhh
memegauste Jun 17, 2021
59baa55
.
memegauste Jun 25, 2021
5b626d3
Django settingsify
memegauste Jul 14, 2021
10d2a1d
Merge remote-tracking branch 'origin/master'
memegauste Jul 14, 2021
449067e
Use all of the CPU cores for compilation
memegauste Jul 18, 2021
4073f83
Fix broken tests message
memegauste Jul 26, 2021
1811fc7
Optional flag for all cores usage
memegauste Aug 4, 2021
6da6e7c
Pardon
memegauste Aug 4, 2021
a6a72ea
Microoptimizations based on fstring, needs testing
memegauste Aug 4, 2021
4ac9d1b
Ops
memegauste Aug 4, 2021
05c21b2
Fix
memegauste Aug 4, 2021
f301fd2
Fix membership performance
memegauste Aug 4, 2021
7f9ef55
django-haystack git home moved
claudep Aug 9, 2021
5934d8d
Dropped Python 2 support
claudep Aug 9, 2021
9751fb0
install_xapian.sh: pin sphinx to avoid compatibility issues with 2.0
asedeno May 6, 2019
12576af
Add CI tests for Django 2.2, 3.0, 3.1
claudep Aug 9, 2021
ef3e219
Simplified super() calls
claudep Aug 9, 2021
b3ff5e5
Drop support for Django < 2.2 and add Django 3.2 support
claudep Aug 22, 2021
da4a0ad
Update Django dependency in requirements.txt
claudep Aug 22, 2021
c69f2c6
install_xapian.sh - unpin sphinx for xapian >= 1.4.12
asedeno Sep 20, 2021
75cfce9
Added builder script xapian_wheel_builder.sh
asedeno Sep 23, 2021
2906116
work on xapian-delve issues
asedeno Sep 23, 2021
893a229
Replace Travis CI by GitHub Actions
asedeno Aug 22, 2021
6827c82
Replaced travis build badge by GH Actions one
claudep Sep 24, 2021
b5b6d37
Ensure README is read with utf-8 encoding in setup.py
claudep Sep 24, 2021
7623860
xapian_wheel_builder - better metadata
asedeno Sep 24, 2021
a1b93fc
Better coverage handling
asedeno Sep 24, 2021
d1f2827
Drop code meant to support Xapian 1.2.x, 1.3.x.
asedeno Sep 25, 2021
2fb5a88
drop _old_xapian_sort
asedeno Sep 26, 2021
80e5cf3
Github Actions - expand test matrix
asedeno Sep 26, 2021
b325b8a
Add a ChangeLog
claudep Oct 26, 2021
f868b1e
Add Changelog to Manifest
claudep Oct 28, 2021
81c288b
Remove force_text usage
claudep Nov 12, 2021
75a5ac3
coveragerc - switch from source to include
asedeno Nov 13, 2021
67d0f84
Bump version to 3.0.1
claudep Nov 13, 2021
cd8e238
Expand the test matrix to include python 3.10 / django 3.2
asedeno Nov 13, 2021
4fc4d90
Revert "coveragerc - switch from source to include"
asedeno Nov 13, 2021
0bb70c1
Dropped support for Python 3.6
claudep Feb 5, 2022
c931e80
Remove obsolete comment backend docstring.
claudep Feb 5, 2022
5f1df4f
tests for non-multiprocessing management commands
ajslater Feb 7, 2022
ec98a86
GitHub Actions: rework test matrix
asedeno Feb 7, 2022
db033d7
Management Command Database Locking
ajslater Feb 10, 2022
40317e3
Bump Xapian version to 1.4.19
asedeno Mar 19, 2022
041d8c6
Merge remote-tracking branch 'from/master'
memegauste Jul 7, 2022
7fa1c54
Fix multiple issues with merge
memegauste Jul 7, 2022
3db84eb
Automatically detect all cores when the --use-all-cores flag is passed
Jul 8, 2022
8ff568e
Merge remote-tracking branch 'origin/master' into configure-jflag
pszczepanski-wt Sep 30, 2022
e665171
Merge
pszczepanski-wt Sep 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion install_xapian.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ if [ -z "$VERSION" ]; then
exit 1
fi

JFLAG=1
if [ "$2" = "--use-all-cores" ]; then
JFLAG=$(($(getconf _NPROCESSORS_ONLN) + 1))
fi
Comment on lines +11 to +14
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to work on current macOS and FreeBSD, though some older FreeBSD may only have NPROCESSORS_ONLN (no leading underscore). That seems to fail in a way that limits us to 1 processor and then proceeds, so that's probably fine.


# prepare
mkdir -p $VIRTUAL_ENV/packages && cd $VIRTUAL_ENV/packages

Expand All @@ -27,7 +32,7 @@ tar xf ${BINDINGS}.tar.xz
# install
echo "Installing Xapian-core..."
cd $VIRTUAL_ENV/packages/${CORE}
./configure --prefix=$VIRTUAL_ENV && make && make install
./configure --prefix=$VIRTUAL_ENV && make -j$JFLAG && make install

PYTHON_FLAG=--with-python3

Expand Down
7 changes: 6 additions & 1 deletion xapian_wheel_builder.sh
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,16 @@ case "${uname_sysname}" in
;;
esac

JFLAG=1
if [ "$2" = "--use-all-cores" ]; then
JFLAG=$(($(getconf _NPROCESSORS_ONLN) + 1))
fi

echo "Building xapian core..."
(
cd "src/${CORE}"
./configure --prefix="${pprefix}"
make
make -j$JFLAG
make install
)

Expand Down