Roman Minkin

Nested queries in MongoDB shell (almost joins)

Sun, 02 Oct 2016 20:57:26 +0000

I’m going to show you how to easily query data across multiple collections in MongoDB shell. Which is almost equal to SQL’s JOIN but not ;)

Assume we have two collections users

> db.users.insert({_id: "usr-1", name: "Tom"})
> db.users.insert({_id: "usr-2", name: "John"})
> db.users.insert({_id: "usr-3", name: "Kate"})

> db.users.find()

{ "_id" : "usr-1", "name" : "Tom" }
{ "_id" : "usr-2", "name" : "John" }
{ "_id" : "usr-3", "name" : "Kate" }

and posts

> db.posts.insert({_id: "pst-1", user_id: "usr-1", body: "body 1", tags: ["apples", "oranges"]})
> db.posts.insert({_id: "pst-2", user_id: "usr-1", body: "body 2", tags: ["apples"]})
> db.posts.insert({_id: "pst-3", user_id: "usr-2", body: "body 3", tags: ["oranges"]})
> db.posts.insert({_id: "pst-4", user_id: "usr-1", body: "body 4", tags: []})
> db.posts.insert({_id: "pst-5", user_id: "usr-3", body: "body 5", tags: ["apples"]})

> db.posts.find()

{ "_id" : "pst-1", "user_id" : "usr-1", "body" : "body 1", "tags" : [ "apples", "oranges" ] }
{ "_id" : "pst-2", "user_id" : "usr-1", "body" : "body 2", "tags" : [ "apples" ] }
{ "_id" : "pst-3", "user_id" : "usr-2", "body" : "body 3", "tags" : [ "oranges" ] }
{ "_id" : "pst-4", "user_id" : "usr-1", "body" : "body 4", "tags" : [ ] }
{ "_id" : "pst-5", "user_id" : "usr-3", "body" : "body 5", "tags" : [ "apples" ] }

…and let’s say we need to find all the users who have at least one post tagged with oranges.

There are quite a few options to do it in half manual way or with MongoDB v3.2 and new $lookup operator from aggregation framework, but here is a pretty simple solution which is easy to remember and it’s a one liner:

db.users.find({_id: {$in: db.posts.find({tags: {$in: ["oranges"]}}).map(function(post){ return post.user_id })}})

{ "_id" : "usr-1", "name" : "Tom" }
{ "_id" : "usr-2", "name" : "John" }

Let’s break it down:

First part is just a standard finding users with _id matching values $in a given array db.users.find({_id: {$in:...})
Second part is a sub query which uses JavaScript map() function on the query cursor and maps results into array of users’ _id (as we know mongodb query does not return an array of objects, but cursor)
Inside the map() functions we specify a callback functions and what field we want to return from every post object. If we run just the second query we would get an array of users’ _id

> db.posts.find({tags: {$in: ["oranges"]}}).map(function(post){ return post.user_id })

[ "usr-1", "usr-2" ]

So basically under the hood our original query becomes this

> db.users.find({_id: {$in: [ "usr-1", "usr-2" ]}})

Another trick is if we have a case when second query users .FindOne() which returns a singular object instead of cursor, we do not need to use map() function and our query becomes a bit simpler

> db.users.find({_id: db.posts.findOne({_id: "pst-5"}).user_id})

{ "_id" : "usr-3", "name" : "Kate" }

Hope you enjoyed it! And keep hacking!

How to remove multiple Docker images

Sat, 10 Oct 2015 20:18:00 +0000

When I’m playing with Docker for a new project I usually end up having dozens or even more of <none> images:

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<none>              <none>              affd29c496e2        3 seconds ago       525.5 MB
<none>              <none>              aeffd5bae1bb        27 seconds ago      526.3 MB
<none>              <none>              55b57705daa5        2 minutes ago       526.3 MB
mongo               3                   e047731b9266        4 days ago          261.3 MB
gliderlabs/alpine   3.2                 2cc966a5578a        3 weeks ago         5.25 MB
golang              1.4                 c5a2538d92ec        4 weeks ago         517.3 MB

It’s pretty annoying to remove images one by one and copy/paste their IDs, so I came up with simple one line command to remove all the <none>:

$ docker rmi -f $(docker images | grep "<none>" | awk '{print $3}')
Deleted: aeffd5bae1bbc107116634cc86a0d07f43ab448fc341e529b910ee85ed4b9c5d
Deleted: 5c7cb2f19237ff87d7f47a770307ec42cac666a9184ec39e0903d4f584b0e4e9
Deleted: 55b57705daa53e579e561244fd3a42f86a504b607c4915e7173e9904cc746f93
Deleted: dabad59a42fd1e7ea32acda815655e64de57d3d20632d24d87e6dfd0cb03d5b9
Deleted: 23a6040841af053dfb0e4a96612f827b5121b4736398bf282ab96240426b71ee

And as you can see no more <none> images:

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
mongo               3                   e047731b9266        4 days ago          261.3 MB
gliderlabs/alpine   3.2                 2cc966a5578a        3 weeks ago         5.25 MB
golang              1.4                 c5a2538d92ec        4 weeks ago         517.3 MB

You can also change filter criteria for grep command to what ever you want to remove.

Or even go further and create an alias in your ~/.bash_profile:

dockerRemoveImagesWithCriteria() {
    if [[ -z "$1" ]]; then
        $1="<none>"
    fi

    count=$(docker images | grep "$1" | wc -l)

    if [[ "$count" -eq 0 ]]; then
        echo "No docker images found for criteria '$1'"
    else
       docker rmi -f $(docker images | grep "$1" | awk '{print $3}')
    fi
}

alias docker-remove=dockerRemoveImagesWithCriteria

…do not forget to source your ~/.bash_profile :

$ source ~/.bash_profile

…and call it from the terminal:

$ docker-remove none
Deleted: 13bb8e2bbc8731802084b496ed9909d9818c58ff219e6043bfbc2cb1fabcd1ad
Deleted: a75b35de499ffd7959bced489e87a83ab3505b6af47e58eb26bf259e468a95d0
Deleted: 3aedddfdd65de9fb2b2ce14378b8cd051fbea775b3959eead24bde551f945b10

Good to see you automation for meetups

Sun, 08 Jun 2014 11:21:29 +0000

After each meetup many of you receive email with the subject Stay in touch! and a link to Meetup.com with simple questionnaire page How was the Meetup?

To keep networking level high I prefer to Say hello to everybody, but i’m too lazy to do it manually otherwise I would not be a developer, so here is a solution!

Meetup.com as many sites uses jQuery library, which means that we can open Developer Console in our browser and run simple JavaScript command which will do all the magic for us:

Go to Meetup.com page with questionnaire form
Open Developer Console in Crome, FireFox, Safari, etc
Run following code:

(function(){
    var i;

    function tryToSayHello() {
        var button = $('p.gtsy > a.D_submit').not('.g2cu')[0],
            next   = $('a[rel=next]');


        if (button) {
            button.click();
            console.log('click');

        } else if (next) {
            next.click();

        } else {
            clearInterval(i);
            console.log('Done!');
        }
    }

    i = setInterval(tryToSayHello, 300);
})()

Enjoy!