MongoDB

MongoDB is a NoSQL database management system based on Javascript using documents similar to JSON.

Getting Started

This will start a new project using your local server and mongoose via node.

Local Machine

Install MongoDB by doing the following[14]:

1. Add the homebrew tap: brew tap mongodb/brew
2. Install MongoDB: brew install mongodb-community.

Once installed, you can start your local server using brew services start mongodb-community (Be sure to stop the server when completed with your work using brew services stop mongodb-community).

In the terminal, start the mongo shell: mongo.

Database

Creating a new database is done when a query is made.

Node

Create a new folder and initialize a new node project using npm: npm init -y. Then install npm i mongoose.

Create a new file called server.js and copy this into the contents:

const mongoose = require('mongoose');                // import Mongoose
mongoose.connect('mongodb://localhost:27017/test', { // create connection to localhost
  useNewUrlParser: true,
  useUnifiedTopology: true
});

const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function() {                           // Inside here is where the action happens
  // Create schema for collection
  const kittySchema = new mongoose.Schema({
    name: String,
    color: {
      type: String,
      enum: ['brown', 'orange'],
    },
  });
  const Cat = mongoose.model('Cat', { name: String });

  const kitty = new Cat({
    name: 'Zildjian',
    color: 'orange'
  });         // Create a new document of Cat
  console.log(kitty.name);                             // 'Zildjian'
  kitty.save()
    .then(() => console.log('meow'));                  // Save the new document in the Cat collection
});

In the terminal, run node server.js. This should log 'Zildjian' along with meow.

Indexes

If your collection does not have an index, your query could take a very long time, as it will have to scan every single document within your collection and analyze it to see if it matches your query.

// create an index in collectionName from highest score to lowest
db.collectionName.createIndex( { score: -1 } )

Import and Export

mongoexport: Export JSON or CSV

$ mongoexport --db=dbName --collection=collectionName --out=filename.json

When you export in CSV format, you must specify the fields in the documents to export.

# for CSV etc.
$ mongoexport --db=dbName --collection=collectionName --fields=name,address --out=filename.json

mongoimport: Import JSON, CSV, or TSV

Use mongoimport to bring in Extended JSON, CSV, or TSV files to your Mongo database. First, start up your database, and then use mongoimport in the command line:

#             use the PetHotel db      use header line as field names
$ mongoimport --db=PetHotel --type=csv --headerline --file=pets.csv

# If no header line is availabe:       list them out after the --fields option
$ mongoimport --db=PetHotel --type=csv --fields=_id,name,type --file=pets.csv

Types

You can specify the types of the columns within the header line of your CSV.

id.int32(),summary.string(),recommend.boolean(),helpfulness.int32()
1,"This product was great!",true,8

$ mongoimport --db=PetHotel --type=csv --headerline --columnsHaveTypes --file=pets.csv

Null Values

If you have null values in your CSV, you can remove these null values between two commas (or after one if at the end of a line) and tell mongoimport to --ignoreBlanks. This won't import null values, but when interacting with your database, you can use the $exists query operator to query for them and $ifNull to cast missing fields as null in an aggregation pipeline.

In the below example, the summary is null, but we still want it to accept strings.

id.int32(),summary.string(),recommend.boolean(),helpfulness.int32()
1,null,true,8

If we were to import this as is, we would end up with a summary field containing an empty string. After replacing the ,null, with ,, and using --ignoreBlanks on import, this field ends up missing, which is the desired behavior.

To query for fields that have a missing of null field:

> db.collection_name.find({ summary: { $exists: false }})

And to get the field back with a null value in the place of that missing field:

> db.collection_name.aggregate([
... // match document with id of 1
... { $match: { id: 1 }},
... // show all fields and cast summary to null if missing
... { $project: {
...   _id: 0,
...   id: 1,
...   summary: { $ifNull: [ '$summary', null ]},
...   recommend: 1,
...   helpfulness: 1
... }}
... ]);

mongodump: Export BSON (native format)

To export a collection or database from a mongo database, use mongodump.

$ mongodump --db=dbName --collection=collectionName --out=path/for/output

mongorestore: Import BSON (native format)

To import a collection or database from a dumped mongo database, use mongorestore.

$ mongorestore -d dbName ./dump/path

Clone Database

You can clone a database on the same server using the following command:

$ mongodump --archive --db=dbName | mongorestore --archive  --nsFrom='dbName.*' --nsTo='newDbName.*'

Data Modification

If you need to modify data that already exists in a document due to a bad import or whatever, you can run a command like this in the mongo shell to change the values in place.

Let's assume a collection of dogs:

[
  {
    "_id": ...,
    "name": "Doggo",
    "goodBoy": "true", // should be a boolean
    "enemy": "null",   // or could be "mailman" or other string
  },
  ...
]

When completed, this will change all goodboy properties to a boolean and if enemy was imported with null as the string, it will convert it into an actual null value. The bulkWrite is done as it is much faster than sending individual requests.

var requests = [];
var cursor = db.dogs.find({});

cursor.forEach(item => {
  requests.push({
    "updateOne": {
      "filter": { "_id": item._id },
      "update": {
        "$set": {
          "goodBoy": Boolean(item.goodboy),
          "enemy": item.enemy === 'null' ? null : item.enemy
        }
      }
    }
  });

  if (requests.length === 1000) {
    // Execute per 1000 operations and re-init
    db.dogs.bulkWrite(requests);
    requests = [];
  }
});

if (requests.length) {
  db.dogs.bulkWrite(requests);
  requests = [];
}

Syntax

References

1. https://www.mongodb.com/basics/examples
2. https://www.mongodb.com/basics/create-database
3. https://docs.mongodb.com/manual/core/databases-and-collections/
4. https://docs.mongodb.com/manual/core/aggregation-pipeline/
5. https://database.guide/import-a-csv-file-into-mongodb-with-mongoimport/
6. https://docs.mongodb.com/manual/indexes/
7. https://stackoverflow.com/questions/37718005/change-document-value-from-string-to-objectid-using-update-query
8. https://www.mongodb.com/developer/how-to/SQL-to-Aggregation-Pipeline/
9. https://docs.mongodb.com/database-tools/mongoimport/#std-label-example-csv-import-types
10. https://docs.mongodb.com/database-tools/mongodump/
11. https://stackoverflow.com/questions/21233290/mongorestore-error-dont-know-what-to-do-with-the-dump-file
12. Schema Design
13. GitHub - mongodb/homebrew-brew: The Official MongoDB Software Homebrew Tap

Last modified: 202401040446