§2024-12-14
$ wget https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2204-arm64-100.10.0.tgz <-- this is for arm64 machine
$ tar xvf src/mongodb-database-tools-ubuntu2204-arm64-100.10.0.tgz
$ sudo cp -v mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/* /usr/local/bin
[sudo] password for alexlai:
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/bsondump' -> '/usr/local/bin/bsondump'
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/mongodump' -> '/usr/local/bin/mongodump'
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/mongoexport' -> '/usr/local/bin/mongoexport'
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/mongofiles' -> '/usr/local/bin/mongofiles'
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/mongoimport' -> '/usr/local/bin/mongoimport'
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/mongorestore' -> '/usr/local/bin/mongorestore'
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/mongostat' -> '/usr/local/bin/mongostat'
'mongodb-database-tools-ubuntu2204-arm64-100.10.0/bin/mongotop' -> '/usr/local/bin/mongotop'
$ wget https://downloads.mongodb.com/compass/mongosh-2.3.6-linux-arm64.tgz
$ tar xvf mongosh-2.3.6-linux-arm64.tgz
$ mongosh-2.3.6-linux-arm64/bin/mongosh --version
2.3.6
$ sudo cp mongosh-2.3.6-linux-arm64/bin/mongosh /usr/local/bin
$ mongosh --version
2.3.6
(base) alexlai@JetsonOrinNano:~/R-proj$ mongoexport --uri="mongodb://siteRootAdmin:b23258585@ysmongov801.yushei.com.tw:27017,ysmongov802.yushei.com.tw:27017,ysmongov803.yushei.com.tw:27017/YuSheiDB?replicaSet=ys20241107&authSource=admin" --collection=car_in_out --out=car_in_out-`date+"%m-%d-%y-%H:%M"`.json --jsonArray
2024-12-14T19:21:03.379+0800 connected to: mongodb://[**REDACTED**]@ysmongov801.yushei.com.tw:27017,ysmongov802.yushei.com.tw:27017,ysmongov803.yushei.com.tw:27017/YuSheiDB?replicaSet=ys20241107&authSource=admin
2024-12-14T19:21:04.380+0800 [........................] YuSheiDB.car_in_out 0/491838 (0.0%)
2024-12-14T19:21:05.381+0800 [........................] YuSheiDB.car_in_out 8000/491838 (1.6%)
2024-12-14T19:21:06.380+0800 [#.......................] YuSheiDB.car_in_out 24000/491838 (4.9%)
2024-12-14T19:21:07.381+0800 [#.......................] YuSheiDB.car_in_out 40000/491838 (8.1%)
2024-12-14T19:21:08.380+0800 [##......................] YuSheiDB.car_in_out 56000/491838 (11.4%)
2024-12-14T19:21:09.380+0800 [###.....................] YuSheiDB.car_in_out 72000/491838 (14.6%)
2024-12-14T19:21:10.380+0800 [###.....................] YuSheiDB.car_in_out 72000/491838 (14.6%)
2024-12-14T19:21:11.380+0800 [###.....................] YuSheiDB.car_in_out 80000/491838 (16.3%)
2024-12-14T19:21:12.380+0800 [####....................] YuSheiDB.car_in_out 96000/491838 (19.5%)
2024-12-14T19:21:13.381+0800 [#####...................] YuSheiDB.car_in_out 112000/491838 (22.8%)
2024-12-14T19:21:14.380+0800 [######..................] YuSheiDB.car_in_out 128000/491838 (26.0%)
2024-12-14T19:21:15.380+0800 [#######.................] YuSheiDB.car_in_out 144000/491838 (29.3%)
2024-12-14T19:21:16.380+0800 [#######.................] YuSheiDB.car_in_out 152000/491838 (30.9%)
2024-12-14T19:21:17.380+0800 [#######.................] YuSheiDB.car_in_out 152000/491838 (30.9%)
2024-12-14T19:21:18.381+0800 [########................] YuSheiDB.car_in_out 168000/491838 (34.2%)
2024-12-14T19:21:19.380+0800 [########................] YuSheiDB.car_in_out 184000/491838 (37.4%)
2024-12-14T19:21:20.381+0800 [#########...............] YuSheiDB.car_in_out 200000/491838 (40.7%)
2024-12-14T19:21:21.380+0800 [##########..............] YuSheiDB.car_in_out 216000/491838 (43.9%)
2024-12-14T19:21:22.380+0800 [###########.............] YuSheiDB.car_in_out 232000/491838 (47.2%)
2024-12-14T19:21:23.380+0800 [###########.............] YuSheiDB.car_in_out 232000/491838 (47.2%)
2024-12-14T19:21:24.381+0800 [###########.............] YuSheiDB.car_in_out 240000/491838 (48.8%)
2024-12-14T19:21:25.380+0800 [############............] YuSheiDB.car_in_out 256000/491838 (52.0%)
2024-12-14T19:21:26.381+0800 [#############...........] YuSheiDB.car_in_out 280000/491838 (56.9%)
2024-12-14T19:21:27.380+0800 [##############..........] YuSheiDB.car_in_out 296000/491838 (60.2%)
2024-12-14T19:21:28.380+0800 [##############..........] YuSheiDB.car_in_out 304000/491838 (61.8%)
2024-12-14T19:21:29.380+0800 [##############..........] YuSheiDB.car_in_out 304000/491838 (61.8%)
2024-12-14T19:21:30.381+0800 [###############.........] YuSheiDB.car_in_out 320000/491838 (65.1%)
2024-12-14T19:21:31.381+0800 [################........] YuSheiDB.car_in_out 336000/491838 (68.3%)
2024-12-14T19:21:32.380+0800 [#################.......] YuSheiDB.car_in_out 352000/491838 (71.6%)
2024-12-14T19:21:33.380+0800 [#################.......] YuSheiDB.car_in_out 368000/491838 (74.8%)
2024-12-14T19:21:34.380+0800 [##################......] YuSheiDB.car_in_out 384000/491838 (78.1%)
2024-12-14T19:21:35.380+0800 [##################......] YuSheiDB.car_in_out 384000/491838 (78.1%)
2024-12-14T19:21:36.381+0800 [###################.....] YuSheiDB.car_in_out 392000/491838 (79.7%)
2024-12-14T19:21:37.381+0800 [####################....] YuSheiDB.car_in_out 416000/491838 (84.6%)
2024-12-14T19:21:38.380+0800 [#####################...] YuSheiDB.car_in_out 432000/491838 (87.8%)
2024-12-14T19:21:39.381+0800 [#####################...] YuSheiDB.car_in_out 448000/491838 (91.1%)
2024-12-14T19:21:40.380+0800 [######################..] YuSheiDB.car_in_out 456000/491838 (92.7%)
2024-12-14T19:21:41.381+0800 [######################..] YuSheiDB.car_in_out 464000/491838 (94.3%)
2024-12-14T19:21:42.380+0800 [#######################.] YuSheiDB.car_in_out 480000/491838 (97.6%)
2024-12-14T19:21:43.233+0800 [########################] YuSheiDB.car_in_out 491839/491838 (100.0%)
2024-12-14T19:21:43.233+0800 exported 491839 records
(base) alexlai@JetsonOrinNano:~/R-proj$ ls
car_in_out-2024-12-14-19:21.json
{"_id":{"$oid":"6715b7bff679da3f6e2a7f26"},"plateText":"BMH5561","inTime":"2024-03-31T21:41:05","outTime":"2024-03-31T23:42:37","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""},
{"_id":{"$oid":"675d86d2118e03ee070c2700"},"plateText":"MYH1959","inTime":"2024-12-14T18:14:49","carType":"motor","inCameraSource":"002","outTime":"2024-12-14T21:37:59","isExit":false,"isMonthlyCar":false,"isUserCreate":false},{"_id":{"$oid":"675d86e3118e03ee070c276e"},"plateText":"BDM9856","inTime":"2024-12-14T16:25:47","carType":"car","inCameraSource":"003","outTime":"2024-12-14T21:38:59","isExit":false,"isMonthlyCar":false,"isUserCreate":false},{"_id":{"$oid":"675d86ff118e03ee070c281f"},"plateText":"NXP0036","inTime":"2024-12-14T19:25:52","carType":"motor","inCameraSource":"002","outTime":"2024-12-14T21:38:59","isExit":false,"isMonthlyCar":false,"isUserCreate":false}]
To use mongoexport with --fieldFile=fields.txt --queryFile=query.json.
plateText
inTime
outTime
carType
inCameraSource
Exit
isMonthlyCar
isUserCreateout
CameraSource
{
"$and": [
{ "isExit": {"$eq": true}},
{ "carType": {"$eq": "car"}},
{ "isMonthlyCar": { "$eq": false}}
]
}
{
"$and": [
{ "isExit": {"$eq": true}},
{ "carType": {"$eq": "motor"}},
{ "isMonthlyCar": { "$eq": false}}
]
}
mongoexport --uri="mongodb://siteRootAdmin:b23258585@ysmongov801.yushei.com.tw:27017,ysmongov802.yushei.com.tw:27017,ysmongov803.yushei.com.tw:27017/YuSheiDB?replicaSet=ys20241107&authSource=admin" --collection=car_in_out --out=car_in_out_car-date +"%m-%d-%y-%H:%M".json --fieldFile=fields.txt --queryFile=carQuery.json --jsonArray
mongoexport --uri="mongodb://siteRootAdmin:b23258585@ysmongov801.yushei.com.tw:27017,ysmongov802.yushei.com.tw:27017,ysmongov803.yushei.com.tw:27017/YuSheiDB?replicaSet=ys20241107&authSource=admin" --collection=car_in_out --out=car_in_out_motor-date +"%m-%d-%y-%H:%M".json --fieldFile=fields.txt --queryFile=motorQuery.json --jsonArray
(PY-3.10.15) alexlai@JetsonOrinNano:~/R-proj/car_in_out$ ls -l car_in_out_*
-rw-rw-r-- 1 alexlai alexlai 34583905 十二 15 09:04 car_in_out_car-12-15-24-09:04.json
-rw-rw-r-- 1 alexlai alexlai 74285869 十二 15 09:05 car_in_out_motor-12-15-24-09:04.json
(PY-3.10.15) alexlai@JetsonOrinNano:~/R-proj/car_in_out$ jq -s 'length' car_in_out_car-12-15-24-09\:04.json
142585
(PY-3.10.15) alexlai@JetsonOrinNano:~/R-proj/car_in_out$ jq -s 'length' car_in_out_motor-12-15-24-09\:04.json
303671
>>> 30 + 31 + 30 + 31 + 31 + 30 + 31 + 30 + 14
258
>>> 142585 / 258
552.6550387596899
>>> 303671 / 258
1177.0193798449613
2024-12-02 ti 2024-12-09
>>> (136+136+138+259+348+727+726)/7
352.85714285714283
>>> (502+428+512+738+876+1672+1498)/7
889.4285714285714
>>>
ChatGPT 4o mini
Log in
Sign up You said: This is my partial json file of a parking lot cars in and out records mongoexport from momgoDB.
{"_id":{"$oid":"6715b7bff679da3f6e2a7f26"},"plateText":"BMH5561","inTime":"2024-03-31T21:41:05","outTime":"2024-03-31T23:42:37","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7bff679da3f6e2a7f29"},"plateText":"RDT3105","inTime":"2024-03-31T21:36:40","outTime":"2024-03-31T23:37:08","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c0f679da3f6e2a8059"},"plateText":"BEW8761","inTime":"2024-03-31T19:37:27","outTime":"2024-03-31T21:54:59","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c0f679da3f6e2a816a"},"plateText":"ASQ6012","inTime":"2024-03-31T18:40:00","outTime":"2024-03-31T21:39:26","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c0f679da3f6e2a8219"},"plateText":"EBA5967","inTime":"2024-03-31T19:03:32","outTime":"2024-03-31T21:24:57","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c1f679da3f6e2a823e"},"plateText":"AJT0915","inTime":"2024-03-31T19:35:16","outTime":"2024-03-31T21:19:13","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c1f679da3f6e2a828a"},"plateText":"AJU1080","inTime":"2024-03-31T18:25:00","outTime":"2024-03-31T21:13:22","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c1f679da3f6e2a830c"},"plateText":"9801ZG","inTime":"2024-03-31T19:09:04","outTime":"2024-03-31T21:01:01","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c1f679da3f6e2a8331"},"plateText":"BRL9263","inTime":"2024-03-31T20:05:47","outTime":"2024-03-31T20:58:24","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c1f679da3f6e2a843a"},"plateText":"3676U8","inTime":"2024-03-31T17:27:17","outTime":"2024-03-31T20:34:37","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c1f679da3f6e2a8440"},"plateText":"BCM6155","inTime":"2024-03-31T18:09:59","outTime":"2024-03-31T20:33:02","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""} {"_id":{"$oid":"6715b7c2f679da3f6e2a84ca"},"plateText":"BVL1986","inTime":"2024-03-31T18:17:24","outTime":"2024-03-31T20:21:45","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""}
How to do use R to summaries the count by day. ChatGPT said: ChatGPT To summarize the count of cars by day using R, you'll need to:
Load your JSON data into R. Parse the inTime field into a date format. Group the data by the date part of the inTime. Count the number of records for each day. Here's how you can do it:
Step 1: Load the necessary libraries r Copy code library(jsonlite) library(dplyr) library(lubridate) jsonlite will help you read and parse your JSON data. dplyr provides functions to manipulate and summarize the data. lubridate helps you handle date and time. Step 2: Load and parse the JSON data If you have your JSON data saved as a file (e.g., parking_lot.json), you can read it using fromJSON() from the jsonlite package.
r Copy code
data <- fromJSON("parking_lot.json", flatten = TRUE) If your data is in a string or directly available, you can use fromJSON() in a similar way.
Step 3: Convert inTime to a Date object Convert the inTime field to a date, extracting only the date part.
r Copy code
data$inDate <- as.Date(data$inTime) Step 4: Summarize the count by day Now that you have the inDate field, you can group the data by day and count the occurrences of each day.
r Copy code
summary_by_day <- data %>% group_by(inDate) %>% summarize(count = n()) %>% arrange(inDate)
print(summary_by_day) This code will:
Group the data by inDate (which is the day part of inTime). Count how many records exist for each day (n()). Sort the results by date. Final Output The output summary_by_day will be a data frame showing the counts of cars entering the parking lot each day.
For example, it might look something like:
bash Copy code
inDate count 1 2024-03-31 12 This table tells you how many cars entered on March 31, 2024 (in this case, 12 cars).
You said:
data <- fromJSON("car_in_out_car-12-15-24-09:04.json", flatten = TRUE) Error in parse_con(txt, bigint_as_char) : parse error: trailing garbage :false,"outCameraSource":""} {"_id":{"$oid":"6715b7bff679da3 (right here) ------^ ChatGPT said: ChatGPT The error you encountered is due to the fact that your JSON file contains multiple JSON records without a proper array wrapper. JSON objects in the file are not enclosed in an array, which makes it invalid for fromJSON() to parse directly.
Here’s how you can handle this issue in R:
Solution 1: Add Square Brackets to Make It a Valid JSON Array Your file appears to contain multiple JSON objects that should be in an array. To fix this:
Manually add square brackets [] around the JSON objects. Separate each object with a comma. For example, your JSON should look like this:
json Copy code [ {"_id":{"$oid":"6715b7bff679da3f6e2a7f26"},"plateText":"BMH5561","inTime":"2024-03-31T21:41:05","outTime":"2024-03-31T23:42:37","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""}, {"_id":{"$oid":"6715b7bff679da3f6e2a7f29"},"plateText":"RDT3105","inTime":"2024-03-31T21:36:40","outTime":"2024-03-31T23:37:08","carType":"car","inCameraSource":"","isExit":true,"isMonthlyCar":false,"isUserCreate":false,"outCameraSource":""}, ... ] Solution 2: Parse It with jsonlite Using a Different Approach If manually editing the JSON file is not feasible, you can also try reading the file and manually adding the array brackets in R:
r Copy code library(jsonlite)
raw_data <- readLines("car_in_out_car-12-15-24-09:04.json")
raw_json <- paste0("[", paste(raw_data, collapse = ","), "]")
data <- fromJSON(raw_json, flatten = TRUE) This will:
Read the raw file line by line. Combine the lines into a single string. Add square brackets around the entire string to form a valid JSON array. Parse it using fromJSON(). After Parsing Once the JSON is correctly loaded into data, you can proceed with the steps to manipulate and summarize the data as described earlier (converting inTime to a Date and counting occurrences by day).
You said:
print(summary_by_day)
inDate count 1 2023-12-31 32 2 2024-01-01 710 3 2024-01-02 178 4 2024-01-03 146 5 2024-01-04 130 6 2024-01-05 238 7 2024-01-06 715 8 2024-01-07 685 9 2024-01-08 176 10 2024-01-09 157
ChatGPT said: ChatGPT To print more rows from a tibble in R, you can use the print() function with the n argument, which specifies how many rows you want to display.
Here’s how you can print more rows:
r Copy code
print(summary_by_day, n = 50) This will display the first 50 rows of the summary_by_day tibble.
If you want to print all rows: You can set n to a larger number, or simply display all rows if you need:
r Copy code
print(summary_by_day, n = Inf) Alternatively, if the dataset is too large and you only need to see a few records for inspection, you can also use the head() or tail() functions to preview the first or last few rows:
r Copy code
head(summary_by_day, 10)
tail(summary_by_day, 10) These functions are helpful when you're working with large datasets and don't want to print the entire data at once.
ChatGPT can make mistakes. Check important info. ?
Return to Top