§2023-07-20
-
orangePi 5 plus, 8G mem, 256PCiSSD
- running Debian Bookworm, can not build mongoDB from source
(PYTHON-3.10.12) alexlai@org5Bookworm:~/build/mongo$ pip install -r etc/pip/compile-requirements.txt
Ignoring pypiwin32: markers 'sys_platform == "win32" and python_version > "3"' don't match your environ
...
pydantic, packaging, gitdb, cffi, anyio, httpcore, gitpython, cryptography, dnspython, pymongo, mongo-tooling-metrics
Attempting uninstall: setuptools
Found existing installation: setuptools 65.5.0
Uninstalling setuptools-65.5.0:
Successfully uninstalled setuptools-65.5.0
DEPRECATION: psutil is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Running setup.py install for psutil ... error
error: subprocess-exited-with-error
× Running setup.py install for psutil did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> psutil
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
[notice] A new release of pip is available: 23.0.1 -> 23.2
[notice] To update, run: pip install --upgrade pip
(PYTHON-3.11.4) alexlai@org5Bookworm:~/build/mongo$ git clone https://github.com/mongodb/mongo.git && cd mongo
(PYTHON-3.11.4) alexlai@org5Bookworm:~/build/mongo$ git checkout r7.1.0-alpha -b test
(PYTHON-3.11.4) alexlai@org5Bookworm:~/build/mongo$ pip install -r etc/pip/compile-requirements.txt
...
Collecting psutil<=5.8.0 (from -r etc/pip/components/core.req (line 2))
Using cached psutil-5.8.0.tar.gz (470 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
§2023-06-29
- orange pi 5 plus 16G mem, 256G PCiSSD
$ source ~/PYTHON-3.11.4/bin/activate
(PYTHON-3.11.4) [alexlai@orangepi5 Python-3.11.4]$ cd ..
(PYTHON-3.11.4) [alexlai@orangepi5 build]$ git clone https://github.com/mongodb/mongo.git
$ git checkout r7.0.0-rc6
(PYTHON-3.11.4) [alexlai@orangepi5 mongo]$ pip install -r etc/pip/compile-requirements.txt
...
Successfully installed Cheetah3-3.2.6.post1 PyYAML-6.0 boto3-1.26.163 botocore-1.29.163 certifi-2023.5.7 cffi-1.15.1 charset-normalizer-2.0.12 cryptography-36.0.2 distro-1.8.0 dnspython-2.3.0 gitdb-4.0.10 gitpython-3.1.31 idna-3.4 jmespath-1.0.1 mongo-ninja-python-1.11.1.5 mongo-tooling-metrics-1.0.8 packaging-21.3 psutil-5.8.0 pycparser-2.21 pydantic-1.10.9 pymongo-4.3.3 pymongo-auth-aws-1.1.0 pyparsing-3.1.0 python-dateutil-2.8.2 regex-2021.11.10 requests-2.26.0 requirements_parser-0.3.1 s3transfer-0.6.1 setuptools-58.5.3 six-1.16.0 smmap-5.0.0 types-PyYAML-6.0.12.10 types-setuptools-57.4.18 typing-extensions-4.7.0 urllib3-1.26.16
(PYTHON-3.11.4) [alexlai@orangepi5 mongo]$ time python3 buildscripts/scons.py install-core -j 8 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors
real 311m42.642s
user 2158m45.138s
sys 96m13.694s
[alexlai@orangepi5 bin]$ ./mongod --version
db version v7.0.0-rc6
Build Info: {
"version": "7.0.0-rc6",
"gitVersion": "ca845553ea3b4b1f54ae3c3fc05a1d3a39cf3bd6",
"openSSLVersion": "OpenSSL 3.0.8 7 Feb 2023",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distarch": "aarch64",
"target_arch": "aarch64"
}
}
(PYTHON-3.11.4) [alexlai@orangepi5 mongo]$ time python3 buildscripts/scons.py install-all-meta -j 8 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors
....Erratum 835769 found and fixed at "build/opt/third_party/IntelRDFPMathLib20U1/libintel_decimal128.a(build/opt/third_party/IntelRDFPMathLib20U1/LIBRARY/src/bid128_div.o)", section 5, offset 0x00005bb8.
Erratum 835769 found and fixed at "build/opt/third_party/IntelRDFPMathLib20U1/libintel_decimal128.a(build/opt/third_party/IntelRDFPMathLib20U1/LIBRARY/src/bid128_scalb.o)", section 2, offset 0x00000328.
Erratum 835769 found and fixed at "build/opt/third_party/IntelRDFPMathLib20U1/libintel_decimal128.a(build/opt/third_party/IntelRDFPMathLib20U1/LIBRARY/src/bid128_string.o)", section 2, offset 0x000000dc.
Erratum 835769 found and fixed at "build/opt/third_party/IntelRDFPMathLib20U1/libintel_decimal128.a(build/opt/third_party/IntelRDFPMathLib20U1/LIBRARY/src/bid64_to_bid128.o)", section 3, offset 0x00000414.
Erratum 835769 found and fixed at "build/opt/third_party/IntelRDFPMathLib20U1/libintel_decimal128.a(build/opt/third_party/IntelRDFPMathLib20U1/LIBRARY/src/bid_round.o)", section 6, offset 0x00000054.
Erratum 835769 found and fixed at "build/opt/third_party/IntelRDFPMathLib20U1/libintel_decimal128.a(build/opt/third_party/IntelRDFPMathLib20U1/LIBRARY/src/bid_round.o)", section 6, offset 0x00000060.
scons: building terminated because of errors.
build/opt/mongo/db/index/db_index_test failed: Error 1
build/opt/mongo/db/repl/db_repl_idempotency_test failed: Error 1
real 230m3.830s
user 1485m4.653s
sys 147m38.664s
(PYTHON-3.11.4) [alexlai@orangepi5 bin]$ ls -l
total 14724192
-rwxr-xr-x 2 alexlai alexlai 123078272 6月 30 00:49 algebra_test
-rwxr-xr-x 2 alexlai alexlai 134764880 6月 30 00:50 bson_mutable_test
-rwxr-xr-x 2 alexlai alexlai 138879456 6月 30 00:49 bson_test
-rwxr-xr-x 2 alexlai alexlai 145752544 6月 30 00:49 bson_util_test
-rwxr-xr-x 2 alexlai alexlai 620960144 6月 30 00:51 client_connpool_integration_test
-rwxr-xr-x 2 alexlai alexlai 620864968 6月 30 00:51 client_dbclient_connection_integration_test
-rwxr-xr-x 2 alexlai alexlai 161388224 6月 30 00:50 client_out_of_line_executor_test
-rwxr-xr-x 2 alexlai alexlai 123003752 6月 30 00:50 concurrent_shared_values_map_test
-rwxr-xr-x 2 alexlai alexlai 178550624 6月 30 00:49 database_name_util_test
-rwxr-xr-x 2 alexlai alexlai 125545144 6月 30 00:49 db_bson_test
-rwxr-xr-x 2 alexlai alexlai 123494736 6月 30 00:50 db_catalog_util_test
-rwxr-xr-x 2 alexlai alexlai 124402520 6月 30 00:50 db_fts_unicode_test
-rwxr-xr-x 2 alexlai alexlai 156086912 6月 30 00:50 db_geo_test
-rwxr-xr-x 2 alexlai alexlai 1249432992 6月 30 00:52 db_percentile_algo_test
-rwxr-xr-x 2 alexlai alexlai 188253864 6月 30 00:50 db_sorter_test
-rwxr-xr-x 2 alexlai alexlai 123378888 6月 30 00:50 executor_stats_test
-rwxr-xr-x 2 alexlai alexlai 237069712 6月 30 00:50 idl_test
-rwxr-xr-x 2 alexlai alexlai 123559504 6月 30 00:50 immutable_test
-rwxr-xr-x 2 alexlai alexlai 15141 6月 29 15:58 install_compass
-rwxr-xr-x 2 alexlai alexlai 9874528 6月 29 23:53 intel_decimal128_readtest
-rwxr-xr-x 2 alexlai alexlai 591 6月 29 23:53 intel_decimal128_readtest_wrapper.py
-rwxr-xr-x 2 alexlai alexlai 114196976 6月 30 00:50 ksdecode
-rwxr-xr-x 2 alexlai alexlai 192427944 6月 30 00:50 logv2_test
-rwxr-xr-x 2 alexlai alexlai 4318094424 6月 29 21:19 mongod
-rwxr-xr-x 2 alexlai alexlai 2424935192 6月 29 21:17 mongos
-rwxr-xr-x 2 alexlai alexlai 178695672 6月 30 00:50 namespace_string_util_test
-rwxr-xr-x 2 alexlai alexlai 136484736 6月 30 00:50 options_parser_test
-rwxr-xr-x 2 alexlai alexlai 167131888 6月 30 00:49 query_datetime_test
-rwxr-xr-x 2 alexlai alexlai 9512687 6月 29 15:58 readtest.in
-rwxr--r-- 2 alexlai alexlai 1782 6月 29 20:27 resmoke.py
-rwxr-xr-x 2 alexlai alexlai 462717408 6月 30 00:51 scripting_test
-rwxr-xr-x 2 alexlai alexlai 302652384 6月 30 00:50 server_description_test
-rwxr-xr-x 2 alexlai alexlai 80258360 6月 30 00:50 set_terminate_dispatch_test
-rwxr-xr-x 2 alexlai alexlai 80316912 6月 30 00:50 set_terminate_from_main_die_in_thread_test
-rwxr-xr-x 2 alexlai alexlai 80318288 6月 30 00:50 set_terminate_from_thread_die_in_main_test
-rwxr-xr-x 2 alexlai alexlai 80330992 6月 30 00:49 set_terminate_from_thread_die_in_thread_test
-rwxr-xr-x 2 alexlai alexlai 80455216 6月 30 00:50 sigaltstack_location_test
-rwxr-xr-x 2 alexlai alexlai 122528608 6月 30 00:50 stacktrace_libunwind_test
-rwxr-xr-x 2 alexlai alexlai 123765008 6月 30 00:50 stacktrace_test
-rwxr-xr-x 2 alexlai alexlai 122311888 6月 30 00:50 stdx_test
-rwxr-xr-x 2 alexlai alexlai 122555360 6月 30 00:50 thread_safety_context_test
-rwxr-xr-x 2 alexlai alexlai 168066184 6月 30 00:50 throughput_probing_test
-rwxr-xr-x 2 alexlai alexlai 301650368 6月 30 00:53 topology_description_test
-rwxr-xr-x 2 alexlai alexlai 301621336 6月 30 00:52 topology_state_machine_test
-rwxr-xr-x 2 alexlai alexlai 123897152 6月 30 00:50 tracing_support_test
-rwxr-xr-x 2 alexlai alexlai 132950920 6月 30 00:50 unittest_test
-rwxr-xr-x 2 alexlai alexlai 123926696 6月 30 00:50 util_net_test
-rwxr-xr-x 2 alexlai alexlai 17300512 6月 29 21:32 wt
(PYTHON-3.11.4) [alexlai@orangepi5 bin]$ strip --strip-unneeded *
strip: install_compass: file format not recognized
strip: intel_decimal128_readtest_wrapper.py: file format not recognized
strip: readtest.in: file format not recognized
strip: resmoke.py: file format not recognized
(PYTHON-3.11.4) [alexlai@orangepi5 bin]$ ls -l mongo*
-rwxr-xr-x 2 alexlai alexlai 122307808 6月 30 04:58 mongod
-rwxr-xr-x 2 alexlai alexlai 88134112 6月 30 04:58 mongos
§2023-06-22
-
Download MongoDB Community ¶Requirements;
-
Ubuntu/Debian - libcurl-dev is provided by three packages. Install one of them.
sudo apt install libcurl4-openssl-dev lzma liblzma-dev
¶2. git clone https://github.com/mongodb/mongo.git
$ git clone https://github.com/mongodb/mongo.git
- 2023-06-23, current release version 6.0.6
$ cd mongo
$ git checkout r6.0.6
$ git branch
* (HEAD detached at r6.0.6)
master
¶3. pip install python requirements
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build/mongo$ pip install -r etc/pip/compile-requirements.txt
...
Installing collected packages: types-setuptools, types-PyYAML, regex, ninja, urllib3, requirements_parser, PyYAML, pyparsing, pymongo, pycparser, psutil, idna, Cheetah3, charset-normalizer, certifi, requests, packaging, cffi, cryptography
DEPRECATION: psutil is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Running setup.py install for psutil ... done
DEPRECATION: Cheetah3 is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Running setup.py install for Cheetah3 ... done
Successfully installed Cheetah3-3.2.6.post1 PyYAML-6.0 certifi-2023.5.7 cffi-1.15.1 charset-normalizer-2.0.12 cryptography-36.0.2 idna-3.4 ninja-1.11.1 packaging-21.3 psutil-5.8.0 pycparser-2.21 pymongo-3.13.0 pyparsing-3.1.0 regex-2021.11.10 requests-2.26.0 requirements_parser-0.3.1 types-PyYAML-6.0.12.10 types-setuptools-57.4.18 urllib3-1.26.16
[notice] A new release of pip is available: 23.0.1 -> 23.1.2
[notice] To update, run: pip install --upgrade pip
$ source ~/PYTHON-3.11.4/bin/activate
(PYTHON-3.11.4) alexlai@orangepi5plus:~/build/mongo$
(PYTHON-3.11.4) alexlai@orangepi5plus:~/build/mongo$ pip install -r etc/pip/compile-requirements.txt
...
Downloading psutil-5.8.0.tar.gz (470 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 470.9/470.9 kB 8.2 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
python-3.11.4 is too new, downgrade it.
¶Note: In order to compile C-based Python modules, you'll also need the Python and OpenSSL C headers. Run:
- Fedora/RHEL - dnf install python3-devel openssl-devel
- Ubuntu (20.04 and newer)/Debian (Bullseye and newer) - apt install python-dev-is-python3 libssl-dev
- Ubuntu (18.04 and older)/Debian (Buster and older) - apt install python3.7-dev libssl-dev
$ sudo apt install python-dev-is-python3 libssl-dev
¶ Compile
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build$ cd mongo/
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build/mongo$ git branch
* (HEAD detached at r6.0.6)
master
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build/mongo$ which python3
/opt/xfs/home/alexlai/PYTHON-3.10.12/bin/python3
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build/mongo$ time python3 buildscripts/scons.py install-core -j 8 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors
...
....
Skipping ranlib for thin archive build/opt/mongo/db/s/libtransaction_coordinator.a
Linking build/opt/mongo/db/mongod
Install file: "build/opt/mongo/s/mongos" as "build/install/bin/mongos"
Install file: "build/opt/mongo/db/mongod" as "build/install/bin/mongod"
scons: done building targets.
real 443m11.562s
user 2212m26.541s
sys 97m44.815s
$ pwd
/opt/xfs/home/alexlai/build/mongo/build/install/bin
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build/mongo/build/install/bin$ ls -l
total 6582108
-rwxr-xr-x 2 alexlai alexlai 4315517480 Jun 22 22:18 mongod
-rwxr-xr-x 2 alexlai alexlai 2424553424 Jun 22 22:07 mongos
-rwxr--r-- 2 alexlai alexlai 1782 Jun 22 14:56 resmoke.py
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build/mongo/build/install/bin$ ./mongod --version
db version v7.0.0-rc5
Build Info: {
"version": "7.0.0-rc5",
"gitVersion": "f568805ae0329f0d4be6a320cde750bc15b9701d",
"openSSLVersion": "OpenSSL 3.0.9 30 May 2023",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distarch": "aarch64",
"target_arch": "aarch64"
}
}
(PYTHON-3.10.12) alexlai@orangepi5plus:~/build/mongo/build/install/bin$ ./mongos --version
mongos version v7.0.0-rc5
Build Info: {
"version": "7.0.0-rc5",
"gitVersion": "f568805ae0329f0d4be6a320cde750bc15b9701d",
"openSSLVersion": "OpenSSL 3.0.9 30 May 2023",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distarch": "aarch64",
"target_arch": "aarch64"
}
}
$ time python3 buildscripts/scons.py install-all-meta -j 8 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors
enerating library build/opt/mongo/s/libsharding_router_test_fixture.a
Skipping ranlib for thin archive build/opt/mongo/s/libsharding_router_test_fixture.a
Linking build/opt/mongo/db/s/db_s_shard_server_test
collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
scons: *** [build/opt/mongo/db/s/db_s_config_server_test] Error 1
collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
scons: *** [build/opt/mongo/db/s/db_s_shard_server_test] Error 1
collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
scons: *** [build/opt/mongo/db/repl/db_repl_test] Error 1
scons: building terminated because of errors.
build/opt/mongo/db/s/db_s_config_server_test failed: Error 1
build/opt/mongo/db/s/db_s_shard_server_test failed: Error 1
build/opt/mongo/db/repl/db_repl_test failed: Error 1
real 445m14.490s
user 1619m11.686s
sys 83m22.462s
alexlai@orangepi5plus:~/build/mongo/build/install/bin$ ls
absl_comparison_bm db_geo_test mongos stacktrace_bm
algebra_test db_integration_test mongotrafficreader stacktrace_libunwind_test
base64_bm db_ops_integration_test namespace_string_util_test stacktrace_test
base_test db_query_collation_test network_interface_ssl_test status_bm
bson_bm db_sorter_test options_parser_test stdx_test
bsoncolumn_bm decimal_counter_bm platform_test storage_key_string_bm
bson_integration_test document_bm query_datetime_test storage_record_id_bm
bson_mutable_test endian_bm readtest.in string_bm
bson_test executor_integration_test replica_set_monitor_integration_test system_resource_canary_bm
bson_util_test executor_stats_test resmoke.py thread_safety_context_test
cancellation_bm fail_point_bm rpc_integration_test throughput_probing_test
client_connpool_integration_test future_bm scoped_db_connection_pool_test ticketholder_bm
client_dbclient_connection_integration_test hash_table_bm scripting_mozjs_test tick_source_bm
client_out_of_line_executor_test idl_test scripting_test topology_description_test
client_rs_test immutable_test sdam_json_test topology_listener_test
clock_source_bm install_compass server_description_test topology_manager_test
commands_bm intel_decimal128_readtest server_selection_json_test topology_state_machine_test
concurrent_shared_values_map_test intel_decimal128_readtest_wrapper.py server_selector_test tracing_support_test
condition_variable_bm itoa_bm set_terminate_dispatch_test transport_integration_test
crypto_bm jws_validator_test set_terminate_from_main_die_in_thread_test unittest_test
crypto_test ksdecode set_terminate_from_thread_die_in_main_test util_net_ssl_test
database_name_util_test lock_manager_bm set_terminate_from_thread_die_in_thread_test util_net_test
db_bson_test logv2_bm shell_test uuid_bm
db_catalog_util_test logv2_test sigaltstack_location_test wt
db_fts_unicode_test mongod simple8b_bm
- failes on x86_64Arch
＄ｔime python3 buildscripts/scons.py install-devcore -j 8 --ssl --linker=gold CFLAGS="-march=armv8-a+crc -mtune=generic" --disable-warnings-as-errors
...
src/mongo/db/free_mon/free_mon_options.cpp:83:59: error: request for member 'getValue' in 'swState', which is of non-class type 'int'
83 | globalFreeMonParams.freeMonitoringState = swState.getValue();
| ^~~~~~~~
scons: *** [build/59f4f0dd/mongo/db/free_mon/free_mon_options.o] Error 1
scons: building terminated because of errors.
build/59f4f0dd/mongo/db/free_mon/free_mon_options.o failed: Error 1
alexlai@orangepi5plus:~/build/mongoBinary$ ls -l
total 8181376
-rwxr-xr-x 1 alexlai alexlai 1215850224 Jun 23 16:01 mongo
-rwxr-xr-x 1 alexlai alexlai 4315517480 Jun 23 16:01 mongod
-rwxr-xr-x 1 alexlai alexlai 2424553424 Jun 23 16:01 mongos
-rwxr-xr-x 1 alexlai alexlai 421917880 Jun 23 16:01 mongotrafficreader
alexlai@orangepi5plus:~/build/mongoBinary$ strip --strip-all *
alexlai@orangepi5plus:~/build/mongoBinary$ ls -l
total 281440
-rwxr-xr-x 1 alexlai alexlai 53618600 Jun 23 16:04 mongo
-rwxr-xr-x 1 alexlai alexlai 122585960 Jun 23 16:04 mongod
-rwxr-xr-x 1 alexlai alexlai 88241768 Jun 23 16:04 mongos
-rwxr-xr-x 1 alexlai alexlai 23737384 Jun 23 16:04 mongotrafficreader