/munetakaJupyterHub/Computer/MongoDB/Sharding/02-TwoNodeShardingServer.md

§2023-08-06

MongoDB Sharding

Each shard is a replica set in and of itself. This process will be similar to the config servers, but using the --shardsvr option. Make sure to use a different replica set name for each shard.

Machine	IP	OS	mongod Version	Replication Set	momgosh Avialbale	Saharding
orgpi5Arch.yushei.net	192.168.48.241	Orange Pi 5 Plus, 16G mem, PCIeSSD 256G Archlinux	7.0.0-rc8	MuneTakaHome:27999	1.10.1 no ssl	No
orgpi5Jammy.yushei.net	192.168.48.247	Orange Pi 5 Plus, 8G mem, PCIeSSD 256G Ubuntu Jammy	7.0.0-rc8	MuneTakaHome:27999	1.10.1 no ssl	No
hc4Lunar.yushei.net	192.168.48.243	Odroid-hc4 Ubuntu Jammy	7.0.0-rc8	MuneTakaHome:27999	1.10.1 no ssl	No
h2nas01.yushei.net	192.168.2.252	odroid-h2, Archlinux	7.0.0-rc8	MuneTakaHome:27999	1.10.1 libcrypto.so.3	No
h2Jammy.yushei.net	59.126.118.193/192.168.11.248	odroid-h2, Ubuntu Jammy	7.0.0-rc8	MuneTakaHome:27999	1.10.1 libcrypto.so.3	No
n2Mnjaro.yushei.net	192.168.48.245	odroid-n2+, ManJaro	7.0.0-rc8	MuneTakaHome:27999	1.1.0 no ssl	No

ShardA

Machine	IP	port	OS	rsplicationSet
n2Mnjaro-01.yushei.net	192.168.48.237	27995	ManJaro	ShardA
hc4bookworm.yushei.net	192.168.48.239	27995	Debian	ShardA
hc4Llunar.yushei.net	192.168.48.243	27995	Ubuntu	ShardA

ShardB

Machine	IP	port	OS	rsplicationSet
n2Mnjaro-01.yushei.net	192.168.48.237	27993	ManJaro	ShardB
hc4bookworm.yushei.net	192.168.48.239	27993	Debian	ShardB
hc4Llunar.yushei.net	192.168.48.243	27993	Ubuntu	ShardB

¶Steps

setup ShardA/B with x.509 but no Security
rs.init(), then use admin to
- db.createUser( {user: "siteRootAdmin",pwd: "b23258585",roles: [ { role: "root", db: "admin" } ]});
then turn on Security

ShardA implementation

n2Mnjaro-01.yushei.net
/etc/mongodb-27995.conf

# mongodb-27995.conf for shard 
# replicationSet: ShardA

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /opt/xfs/mongodb/log/mongod-27995.log

# Where and how to store data.
storage:
  dbPath: /opt/xfs/mongodb/data-27995
  
# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27995
  bindIp: 0.0.0.0   # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.
  
  tls:
      mode: requireTLS
      certificateKeyFile: /opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem
      CAFile: /opt/xfs/mongodb/x.509/mongoCA.crt

# security:
  # authorization: enabled
  # keyFile: /opt/xfs/mongodb/x.509/MuneTakaHomeKey
  # clusterAuthMode: x509  

#operationProfiling:

replication:
  replSetName: "ShardA"

sharding:
  clusterRole: "shardsvr"

## Enterprise-Only Options

#auditLog:

#snmp:

/etc/systemd/system/Mongodb-27995.service

[Unit]
Description=MongoDB Database Server, port 27995, replicationSet: ShardA
Documentation=https://docs.mongodb.org/manual
After=network-online.target
Wants=network-online.target

[Service]
TimeoutStartSec=infinity
User=mongodb
Group=mongodb
Environment="OPTIONS=-f /etc/mongodb-27995.conf"
Environment="MONGODB_CONFIG_OVERRIDE_NOFORK=1"
ExecStart=/usr/local/bin/mongod $OPTIONS
RuntimeDirectory=mongodb
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for mongod as specified in
# https://docs.mongodb.com/manual/reference/ulimit/#recommended-ulimit-settings

[Install]
WantedBy=multi-user.target

startMongosh-27995.sh

#!/bin/bash
# this is to connect shardA

mongosh "mongodb://hc4bookworm.yushei.net:27995/?\
tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/hc4bookworm.yushei.net.pem\
&tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# mongosh "mongodb://siteRootAdmin:b23258585@orgpi5arch.yushei.net:27999,n2mnjaro-01.yushei.net:27999/admin?\
# tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
# &tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# no longer suppoert relicaSet=MuneTakaHome"

console.log("Hello from /hc4bookworm.yushei.net:27995");

rs.init()

Current Mongosh Log ID: 64cf92f2999251e3a420d42a
Connecting to:          mongodb://hc4lunar.yushei.net:27995/?tls=true&tlsCertificateKeyFile=%2Fopt%2Fxfs%2Fmongodb%2Fx.509%2Fhc4lunar.yushei.net.pem&tlsCAFile=%2Fopt%2Fxfs%2Fmongodb%2Fx.509%2FmongoCA.crt&directConnection=true&appName=mongosh+1.10.1
Using MongoDB:          7.0.0-rc8
Using Mongosh:          1.10.1

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

------
   The server generated these startup warnings when booting
   2023-08-06T20:08:35.005+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
   2023-08-06T20:08:35.006+08:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
------

test> rs.initiate( {
...    _id : "ShardA",
...    members: [
...       { _id: 0, host: "n2Mnjaro-01.yushei.net:27995" },
...       { _id: 1, host: "hc4Bookworm.yushei.net:27995" },
...       { _id: 2, host: "hc4Lunar.yushei.net:27995" }
...    ]
... })
{ ok: 1 }
test>

rs.initiate( {
   _id : "ShardA",
   members: [
      { _id: 0, host: "n2Mnjaro-01.yushei.net:27995" },
      { _id: 1, host: "hc4Bookworm.yushei.net:27995" },
      { _id: 2, host: "hc4Lunar.yushei.net:27995" }
   ]
});

ShardA [direct: primary] test> use admin
switched to db admin
ShardA [direct: primary] admin> db.createUser( {
... user: "siteRootAdmin",
... pwd: "b23258585",
... roles: [ { role: "root", db: "admin" } ]
... });
{
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1691326047, i: 4 }),
    signature: {
      hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
      keyId: Long("0")
    }
  },
  operationTime: Timestamp({ t: 1691326047, i: 4 })
}
ShardA [direct: primary] admin>

turn on access control

uncomment

security:
  authorization: enabled
  keyFile: /opt/xfs/mongodb/x.509/MuneTakaHomeKey

startMongosh-29775.sh

[alexlai@n2Mnjaro-01 ~]$ cat ~/scripts/startMongosh-27995.sh 
#!/bin/bash
# this is to connect shardA

# mongosh "mongodb://n2mnjaro-01.yushei.net:27995/?\
# tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
# &tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

mongosh "mongodb://siteRootAdmin:b23258585@\
n2mnjaro-01.yushei.net:27995,hc4bookworm.yushei.net:27995,hc4lunar.yushei.net:27995/admin?\
tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
&tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# no longer suppoert relicaSet=MuneTakaHome"

console.log("Hello from /n2mnjaro-01.yushei.net:27995");

ShardA implementation

n2Mnjaro-01.yushei.net
/etc/mongodb-27995.conf

# mongodb-27995.conf for shard 
# replicationSet: ShardA

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /opt/xfs/mongodb/log/mongod-27995.log

# Where and how to store data.
storage:
  dbPath: /opt/xfs/mongodb/data-27995
  
# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27995
  bindIp: 0.0.0.0   # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.
  
  tls:
      mode: requireTLS
      certificateKeyFile: /opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem
      CAFile: /opt/xfs/mongodb/x.509/mongoCA.crt

# security:
  # authorization: enabled
  # keyFile: /opt/xfs/mongodb/x.509/MuneTakaHomeKey
  # clusterAuthMode: x509  

#operationProfiling:

replication:
  replSetName: "ShardA"

sharding:
  clusterRole: "shardsvr"

## Enterprise-Only Options

#auditLog:

#snmp:

/etc/systemd/system/Mongodb-27995.service

[Unit]
Description=MongoDB Database Server, port 27995, replicationSet: ShardA
Documentation=https://docs.mongodb.org/manual
After=network-online.target
Wants=network-online.target

[Service]
TimeoutStartSec=infinity
User=mongodb
Group=mongodb
Environment="OPTIONS=-f /etc/mongodb-27995.conf"
Environment="MONGODB_CONFIG_OVERRIDE_NOFORK=1"
ExecStart=/usr/local/bin/mongod $OPTIONS
RuntimeDirectory=mongodb
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for mongod as specified in
# https://docs.mongodb.com/manual/reference/ulimit/#recommended-ulimit-settings

[Install]
WantedBy=multi-user.target

startMongosh-27995.sh

#!/bin/bash
# this is to connect shardA

mongosh "mongodb://hc4bookworm.yushei.net:27995/?\
tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/hc4bookworm.yushei.net.pem\
&tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# mongosh "mongodb://siteRootAdmin:b23258585@orgpi5arch.yushei.net:27999,n2mnjaro-01.yushei.net:27999/admin?\
# tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
# &tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# no longer suppoert relicaSet=MuneTakaHome"

console.log("Hello from /hc4bookworm.yushei.net:27995");

rs.init()

Current Mongosh Log ID: 64cf92f2999251e3a420d42a
Connecting to:          mongodb://hc4lunar.yushei.net:27995/?tls=true&tlsCertificateKeyFile=%2Fopt%2Fxfs%2Fmongodb%2Fx.509%2Fhc4lunar.yushei.net.pem&tlsCAFile=%2Fopt%2Fxfs%2Fmongodb%2Fx.509%2FmongoCA.crt&directConnection=true&appName=mongosh+1.10.1
Using MongoDB:          7.0.0-rc8
Using Mongosh:          1.10.1

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

------
   The server generated these startup warnings when booting
   2023-08-06T20:08:35.005+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
   2023-08-06T20:08:35.006+08:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
------

test> rs.initiate( {
...    _id : "ShardA",
...    members: [
...       { _id: 0, host: "n2Mnjaro-01.yushei.net:27995" },
...       { _id: 1, host: "hc4Bookworm.yushei.net:27995" },
...       { _id: 2, host: "hc4Lunar.yushei.net:27995" }
...    ]
... })
{ ok: 1 }
test>

rs.initiate( {
   _id : "ShardA",
   members: [
      { _id: 0, host: "n2Mnjaro-01.yushei.net:27995" },
      { _id: 1, host: "hc4Bookworm.yushei.net:27995" },
      { _id: 2, host: "hc4Lunar.yushei.net:27995" }
   ]
});

ShardA [direct: primary] test> use admin
switched to db admin
ShardA [direct: primary] admin> db.createUser( {
... user: "siteRootAdmin",
... pwd: "b23258585",
... roles: [ { role: "root", db: "admin" } ]
... });
{
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1691326047, i: 4 }),
    signature: {
      hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
      keyId: Long("0")
    }
  },
  operationTime: Timestamp({ t: 1691326047, i: 4 })
}
ShardA [direct: primary] admin>

turn on access control

uncomment

security:
  authorization: enabled
  keyFile: /opt/xfs/mongodb/x.509/MuneTakaHomeKey

startMongosh-29775.sh

[alexlai@n2Mnjaro-01 ~]$ cat ~/scripts/startMongosh-27995.sh 
#!/bin/bash
# this is to connect shardA

# mongosh "mongodb://n2mnjaro-01.yushei.net:27995/?\
# tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
# &tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

mongosh "mongodb://siteRootAdmin:b23258585@\
n2mnjaro-01.yushei.net:27995,hc4bookworm.yushei.net:27995,hc4lunar.yushei.net:27995/admin?\
tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
&tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# no longer suppoert relicaSet=MuneTakaHome"

console.log("Hello from /n2mnjaro-01.yushei.net:27995");

ShardB implementation

n2Mnjaro-01.yushei.net:27993
/etc/mongodb-27993.conf

# mongodb-27995.conf for shard 
# replicationSet: ShardB

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /opt/xfs/mongodb/log/mongod-27993.log

# Where and how to store data.
storage:
  dbPath: /opt/xfs/mongodb/data-27993
  
# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27993
  bindIp: 0.0.0.0   # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.
  
  tls:
      mode: requireTLS
      certificateKeyFile: /opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem
      CAFile: /opt/xfs/mongodb/x.509/mongoCA.crt

# security:
  # authorization: enabled
  # keyFile: /opt/xfs/mongodb/x.509/MuneTakaHomeKey
  # clusterAuthMode: x509  

#operationProfiling:

replication:
  replSetName: "ShardB"

sharding:
  clusterRole: "shardsvr"

## Enterprise-Only Options

#auditLog:

#snmp:

/etc/systemd/system/Mongodb-27993.service

[Unit]
Description=MongoDB Database Server, port 27993, replicationSet: ShardB
Documentation=https://docs.mongodb.org/manual
After=network-online.target
Wants=network-online.target

[Service]
TimeoutStartSec=infinity
User=mongodb
Group=mongodb
Environment="OPTIONS=-f /etc/mongodb-27993.conf"
Environment="MONGODB_CONFIG_OVERRIDE_NOFORK=1"
ExecStart=/usr/local/bin/mongod $OPTIONS
RuntimeDirectory=mongodb
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for mongod as specified in
# https://docs.mongodb.com/manual/reference/ulimit/#recommended-ulimit-settings

[Install]
WantedBy=multi-user.target

startMongosh-27993.sh

#!/bin/bash
# this is to connect shardA

# connect before replicationSet was initiated
mongosh "mongodb://n2mnjaro-01.yushei.net:27993/?\
tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
&tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# mongosh "mongodb://siteRootAdmin:b23258585@\
# n2mnjaro-01.yushei.net:27993,hc4bookworm.yushei.net:27993,hc4lunar.yushei.net:27993/admin?\
# tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
# &tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# no longer suppoert relicaSet=MuneTakaHome"

console.log("Hello from /n2mnjaro-01.yushei.net:27993")
``

- rs.init()

```bash
[alexlai@n2Mnjaro-01 scripts]$ ./startMongosh-27993.sh 
Current Mongosh Log ID: 64d01a8b4e2f1106459bf20f
Connecting to:          mongodb://n2mnjaro-01.yushei.net:27993/?tls=true&tlsCertificateKeyFile=%2Fopt%2Fxfs%2Fmongodb%2Fx.509%2Fn2mnjaro-01.yushei.net.pem&tlsCAFile=%2Fopt%2Fxfs%2Fmongodb%2Fx.509%2FmongoCA.crt&directConnection=true&appName=mongosh+1.10.1
Using MongoDB:          7.0.0-rc8
Using Mongosh:          1.10.1

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

------
   The server generated these startup warnings when booting
   2023-08-07T06:01:08.938+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
------

test> 

test> rs.initiate( {
...    _id : "ShardB",
...    members: [
...       { _id: 0, host: "n2Mnjaro-01.yushei.net:27993" },
...       { _id: 1, host: "hc4Bookworm.yushei.net:27993" },
...       { _id: 2, host: "hc4Lunar.yushei.net:27993" }
...    ]
... })
{ ok: 1 }
test>

rs.initiate( {
   _id : "ShardB",
   members: [
      { _id: 0, host: "n2Mnjaro-01.yushei.net:27993" },
      { _id: 1, host: "hc4Bookworm.yushei.net:27993" },
      { _id: 2, host: "hc4Lunar.yushei.net:27993" }
   ]
});

ShardA [direct: primary] test> use admin
switched to db admin
ShardA [direct: primary] admin> db.createUser( {
... user: "siteRootAdmin",
... pwd: "b23258585",
... roles: [ { role: "root", db: "admin" } ]
... });

db.createUser( {
    user: "siteRootAdmin",
    pwd: "b23258585",
    roles: [ { role: "root", db: "admin" } ]
});

{
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1691360094, i: 4 }),
    signature: {
      hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
      keyId: Long("0")
    }
  },
  operationTime: Timestamp({ t: 1691360094, i: 4 })
}
ShardB [direct: primary] test>

turn on access control

uncomment

security:
  authorization: enabled
  keyFile: /opt/xfs/mongodb/x.509/MuneTakaHomeKey

startMongosh-29775.sh

[alexlai@n2Mnjaro-01 ~]$ cat ~/scripts/startMongosh-27995.sh 
#!/bin/bash
# this is to connect shardA

# connect before replicationSet was initiated
# mongosh "mongodb://n2mnjaro-01.yushei.net:27993/?\
# tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
# &tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

mongosh "mongodb://siteRootAdmin:b23258585@\
n2mnjaro-01.yushei.net:27993,hc4bookworm.yushei.net:27993,hc4lunar.yushei.net:27993/admin?\
tls=true&tlsCertificateKeyFile=/opt/xfs/mongodb/x.509/n2mnjaro-01.yushei.net.pem\
&tlsCAFile=/opt/xfs/mongodb/x.509/mongoCA.crt"

# no longer suppoert relicaSet=MuneTakaHome"

# console.log("Hello from /n2mnjaro-01.yushei.net:27993");

ShardB [direct: secondary] admin> direct means direct connect into this server hardB [primary] admin>, is connected into replicaSet ShardB

4. Configure and Turn On Sharding for the Database