Discussion:
MongoDB: console commands? query embedded indexed fields?
(too old to reply)
Sabine Manaa
2015-08-04 05:08:55 UTC
Permalink
Hi,

I have 2 questions concerning use of mongoDB classes:
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index

Model:
I have trips and each trip has an embedded collection of N receipts like
this:
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}

Index:
For quick search, I created an index on the embeded[1] receipts with this
command at the mongoDB console in a terminal (NOT in smalltalk)
(mongo console starts with mongo in the MongoDB/bin dir):

db.Trips.createIndex({"receipts.receiptDescription":1})

with a command like this in the mongoDB console

db.Trips.find({"receipts.receiptDescription":"receipt 1"})

I get the resulting receipt.

Before, I have to set the database with "use {databasename}" in thr mongodb
console

My questions:
1) how can I send mongo console commands directly from smalltalk, e.g. use
database, set index etc.
Is this possible?
I did not succeed with the mongo command: method, it always returns "no such
cmd".

2) how can I search for a receipt which is indexed as described from
smalltalk?
I did not succeed with the description [2] I am not sure if it is covered by
it.

Regards
Sabine


[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries



--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Norbert Hartl
2015-08-04 08:44:19 UTC
Permalink
Sabine,
Post by Sabine Manaa
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with this
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr mongodb
console
1) how can I send mongo console commands directly from smalltalk, e.g. use
database, set index etc.
I don't know why you want to use "use database". For queries you don't need this because you choose the database and collection by using the smalltalk classes. You can look there if something you need exists.
Creating indexes is not yet supported. I want to do that myself. As always there seems to be to less people implementing stuff like that.
Post by Sabine Manaa
Is this possible?
I did not succeed with the mongo command: method, it always returns "no such
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple

Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary

should do. You can use just the nested properties form. The index will be selected automatically. If you are not sure about queries read about explain [1] to optimize that stuff.

[1] http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain <http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain>

Norbert
Post by Sabine Manaa
I did not succeed with the description [2] I am not sure if it is covered by
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries
--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Sabine Manaa
2015-08-04 12:47:45 UTC
Permalink
Hi Norbert,

I didn't want to use "use database" for queries but for defining the index
from within smalltalk. But if creating indexes is not supported, I will set
the indexes within the mongo console.

The explain option is very helpful I did not know this.

Your answer was very helpful for me, thanks a lot!
Sabine



2015-08-04 10:42 GMT+02:00 NorbertHartl [via Smalltalk] <
Post by Norbert Hartl
Sabine,
Am 04.08.2015 um 07:08 schrieb Sabine Manaa <[hidden email]
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with this
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr mongodb
console
1) how can I send mongo console commands directly from smalltalk, e.g. use
database, set index etc.
I don't know why you want to use "use database". For queries you don't
need this because you choose the database and collection by using the
smalltalk classes. You can look there if something you need exists.
Creating indexes is not yet supported. I want to do that myself. As always
there seems to be to less people implementing stuff like that.
Is this possible?
I did not succeed with the mongo command: method, it always returns "no such
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
should do. You can use just the nested properties form. The index will be
selected automatically. If you are not sure about queries read about
explain [1] to optimize that stuff.
[1]
http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain
Norbert
I did not succeed with the description [2] I am not sure if it is covered by
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries
--
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
------------------------------
If you reply to this email, your message will be added to the discussion
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html
To start a new topic under Pharo Smalltalk Users, email
To unsubscribe from MongoDB: console commands? query embedded indexed
fields?, click here
<http://forum.world.st/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4840882&code=bWFuYWEuc2FiaW5lQGdtYWlsLmNvbXw0ODQwODgyfC0xOTE3OTcxOTg5>
.
NAML
<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840920.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Sabine Manaa
2015-08-06 12:49:39 UTC
Permalink
Norbert,

I have to come back to your answer.

With the .explain option, I can see, how performance is with and without
index on the _mongo console_.
As an example, I see: with index only 2 objects scanned, without: 200.
Performance is perfect with index.

But _from smalltalk_, it seems to iterate over all objects, also with the
index set. Slow.

You wrote: "The index will be selected automatically." It seems that it
does not.

My question:
MongoQuery has an attribute named "where" which is filled with the
mentioned "receipts.receiptDescription".
Because of the name of this attribute "where" and the slow performance, I
assume that the mongo classes use the $where command:

http://docs.mongodb.org/manual/reference/operator/query/where/

Is this right?

In this case with the statement
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' }
asDictionary
the index would not be set automatically.

"The $where provides greater flexibility, but requires that the database
processes the JavaScript expression or function for each document in the
collection. "

Then my question is if the mongo classes support database queries with
index and if yes, how.

I am very interested in your opinion and I think other users should have
the same requirements (querying indexed variables) - how did they solve
this?

Regards
Sabine
Post by Sabine Manaa
Hi Norbert,
I didn't want to use "use database" for queries but for defining the
index from within smalltalk. But if creating indexes is not supported, I
will set the indexes within the mongo console.
Post by Sabine Manaa
The explain option is very helpful I did not know this.
Your answer was very helpful for me, thanks a lot!
Sabine
Post by Norbert Hartl
Sabine,
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with this
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr mongodb
console
1) how can I send mongo console commands directly from smalltalk, e.g. use
database, set index etc.
I don't know why you want to use "use database". For queries you don't
need this because you choose the database and collection by using the
smalltalk classes. You can look there if something you need exists.
Post by Sabine Manaa
Post by Norbert Hartl
Creating indexes is not yet supported. I want to do that myself. As
always there seems to be to less people implementing stuff like that.
Post by Sabine Manaa
Post by Norbert Hartl
Is this possible?
I did not succeed with the mongo command: method, it always returns "no such
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
should do. You can use just the nested properties form. The index will
be selected automatically. If you are not sure about queries read about
explain [1] to optimize that stuff.
Post by Sabine Manaa
Post by Norbert Hartl
[1]
http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain
Post by Sabine Manaa
Post by Norbert Hartl
Norbert
I did not succeed with the description [2] I am not sure if it is covered by
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
Post by Sabine Manaa
Post by Norbert Hartl
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries
--
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Post by Sabine Manaa
Post by Norbert Hartl
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
________________________________
If you reply to this email, your message will be added to the discussion
below:
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html
Post by Sabine Manaa
Post by Norbert Hartl
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed
fields?, click here.
Post by Sabine Manaa
Post by Norbert Hartl
NAML
________________________________
View this message in context: Re: MongoDB: console commands? query
embedded indexed fields?
Post by Sabine Manaa
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
p***@highoctane.be
2015-08-06 13:23:15 UTC
Permalink
Hi Sabine,

Not related to these indexes per se but I had performance problems
associated with the fact that MongoTalk asked for all collections again and
again.

I cached them (in my own MongoTalk version) and added debugging to the
MongoOperations so that I could see what was going on.

This alone sped up the thing by a huge factor. My collections have more
than 20-30K entries and I am querying them extensively. No perf problem
even without the indexes.

How many entries are you scanning?

Phil
Post by Sabine Manaa
Norbert,
I have to come back to your answer.
With the .explain option, I can see, how performance is with and without
index on the _mongo console_.
As an example, I see: with index only 2 objects scanned, without: 200.
Performance is perfect with index.
But _from smalltalk_, it seems to iterate over all objects, also with the
index set. Slow.
You wrote: "The index will be selected automatically." It seems that it
does not.
MongoQuery has an attribute named "where" which is filled with the
mentioned "receipts.receiptDescription".
Because of the name of this attribute "where" and the slow performance, I
http://docs.mongodb.org/manual/reference/operator/query/where/
Is this right?
In this case with the statement
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
the index would not be set automatically.
"The $where provides greater flexibility, but requires that the database
processes the JavaScript expression or function for each document in the
collection. "
Then my question is if the mongo classes support database queries with
index and if yes, how.
I am very interested in your opinion and I think other users should have
the same requirements (querying indexed variables) - how did they solve
this?
Regards
Sabine
Post by Sabine Manaa
Hi Norbert,
I didn't want to use "use database" for queries but for defining the
index from within smalltalk. But if creating indexes is not supported, I
will set the indexes within the mongo console.
Post by Sabine Manaa
The explain option is very helpful I did not know this.
Your answer was very helpful for me, thanks a lot!
Sabine
Post by Norbert Hartl
Sabine,
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with
this
Post by Sabine Manaa
Post by Norbert Hartl
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr
mongodb
Post by Sabine Manaa
Post by Norbert Hartl
console
1) how can I send mongo console commands directly from smalltalk, e.g.
use
Post by Sabine Manaa
Post by Norbert Hartl
database, set index etc.
I don't know why you want to use "use database". For queries you don't
need this because you choose the database and collection by using the
smalltalk classes. You can look there if something you need exists.
Post by Sabine Manaa
Post by Norbert Hartl
Creating indexes is not yet supported. I want to do that myself. As
always there seems to be to less people implementing stuff like that.
Post by Sabine Manaa
Post by Norbert Hartl
Is this possible?
I did not succeed with the mongo command: method, it always returns "no
such
Post by Sabine Manaa
Post by Norbert Hartl
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' }
asDictionary
Post by Sabine Manaa
Post by Norbert Hartl
should do. You can use just the nested properties form. The index will
be selected automatically. If you are not sure about queries read about
explain [1] to optimize that stuff.
Post by Sabine Manaa
Post by Norbert Hartl
[1]
http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain
Post by Sabine Manaa
Post by Norbert Hartl
Norbert
I did not succeed with the description [2] I am not sure if it is
covered by
Post by Sabine Manaa
Post by Norbert Hartl
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
Post by Sabine Manaa
Post by Norbert Hartl
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries
--
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Post by Sabine Manaa
Post by Norbert Hartl
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
________________________________
If you reply to this email, your message will be added to the
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html
Post by Sabine Manaa
Post by Norbert Hartl
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed
fields?, click here.
Post by Sabine Manaa
Post by Norbert Hartl
NAML
________________________________
View this message in context: Re: MongoDB: console commands? query
embedded indexed fields?
Post by Sabine Manaa
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Norbert Hartl
2015-08-06 13:38:50 UTC
Permalink
Sabine,
Post by Sabine Manaa
Norbert,
I have to come back to your answer.
With the .explain option, I can see, how performance is with and without index on the _mongo console_.
As an example, I see: with index only 2 objects scanned, without: 200. Performance is perfect with index.
But _from smalltalk_, it seems to iterate over all objects, also with the index set. Slow.
You wrote: "The index will be selected automatically." It seems that it does not.
Usually it does.
Post by Sabine Manaa
MongoQuery has an attribute named "where" which is filled with the mentioned "receipts.receiptDescription".
http://docs.mongodb.org/manual/reference/operator/query/where/ <http://docs.mongodb.org/manual/reference/operator/query/where/>
Is this right?
In this case with the statement
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
the index would not be set automatically.
"The $where provides greater flexibility, but requires that the database processes the JavaScript expression or function for each document in the collection. "
Then my question is if the mongo classes support database queries with index and if yes, how.
I am very interested in your opinion and I think other users should have the same requirements (querying indexed variables) - how did they solve this?
The handling of indexes needs to be automatic. The rule of thumb is that if you use where clause like b = , c = , a= than you need an compound index in the same order being index(b,c,a). This is handled by the database when it analyzes the query it tries to find an appropriate index. Or seeing it from the other side: If you issue a query with index and the index does not fit the query it is a problem anyway. Are you sure you have only this one thing in the where query?

I use e.g.

COInstalledApp selectOne: {
'device.id' -> aString.
'application.__id' -> anApplication voyageId } asDictionary

on a collection that has approx. 1 Mio. entires and it is fast. I didn't check with explain but I'll do if I find time :)

If I were you I would step in the debugger down to the Voyage part after serialization. Then you have the exact Dictionary that is used to send to mongo. You can then use this query to do an "explain" on it and that should show you something. Another thing you can try (although I don't think makes any difference) is to use the non-javascript query variant. That would be

Trip selectMany: {
'receipts' -> {
'receiptDescription' -> 'Tankquittung' } asDictionary } asDictionary

Norbert
Post by Sabine Manaa
Regards
Sabine
Post by Sabine Manaa
Hi Norbert,
I didn't want to use "use database" for queries but for defining the index from within smalltalk. But if creating indexes is not supported, I will set the indexes within the mongo console.
The explain option is very helpful I did not know this.
Your answer was very helpful for me, thanks a lot!
Sabine
Post by Norbert Hartl
Sabine,
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with this
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr mongodb
console
1) how can I send mongo console commands directly from smalltalk, e.g. use
database, set index etc.
I don't know why you want to use "use database". For queries you don't need this because you choose the database and collection by using the smalltalk classes. You can look there if something you need exists.
Creating indexes is not yet supported. I want to do that myself. As always there seems to be to less people implementing stuff like that.
Is this possible?
I did not succeed with the mongo command: method, it always returns "no such
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
should do. You can use just the nested properties form. The index will be selected automatically. If you are not sure about queries read about explain [1] to optimize that stuff.
[1] http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain <http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain>
Norbert
I did not succeed with the description [2] I am not sure if it is covered by
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents <http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents>
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries <http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries>
--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html <http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html>
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
________________________________
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html <http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html>
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here.
NAML
________________________________
View this message in context: Re: MongoDB: console commands? query embedded indexed fields?
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Sabine Manaa
2015-08-07 12:53:34 UTC
Permalink
Hi Norbert, Hi Phil,

thanks for your help! Now I know what the problem is/was. :-)))

The query was right, the index was used, all is fine.

BUT:
I used an inspector to look at the query's result and opening the inspector
forced to load all the other objects which belong to the object found.
Explanation: I have a person which has N trips with N receipts, all with
back pointers. Opening an Inspector with one trip loads its person and all
other trips of this person hahaha. The magritte description isLazy but the
inspector forced the loading of the objects in the VOMongoRepository at
first time after image starting.

Time millisecondsToRun: [(RKATrip selectMany: {
'receipts.receiptDescription' -> 'a' } asDictionary) size] 764 .
Time millisecondsToRun: [(RKATrip selectMany: {
'receipts.receiptDescription' -> 'a' } asDictionary) inspect] 17234

So, I messed around with myself.

Currently I do not have so much data but I hope in future there will be a
lot and I want the application prepared for that.

BTW: I found this nice document:
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/ws/Voyage/Voyage.pdf
I was mentioned there, thanks! Esteban, if you want, you could change my
name, I married and changed my name to Manaa.

have a nice weekend
Sabine

2015-08-06 15:36 GMT+02:00 NorbertHartl [via Smalltalk] <
Post by Norbert Hartl
Sabine,
Am 06.08.2015 um 14:49 schrieb Sabine Manaa <[hidden email]
Norbert,
I have to come back to your answer.
With the .explain option, I can see, how performance is with and without
index on the _mongo console_.
As an example, I see: with index only 2 objects scanned, without: 200.
Performance is perfect with index.
But _from smalltalk_, it seems to iterate over all objects, also with the index set. Slow.
You wrote: "The index will be selected automatically." It seems that it does not.
Usually it does.
MongoQuery has an attribute named "where" which is filled with the
mentioned "receipts.receiptDescription".
Because of the name of this attribute "where" and the slow performance, I
http://docs.mongodb.org/manual/reference/operator/query/where/
Is this right?
In this case with the statement
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
the index would not be set automatically.
"The $where provides greater flexibility, but requires that the database
processes the JavaScript expression or function for each document in the
collection. "
Then my question is if the mongo classes support database queries with
index and if yes, how.
I am very interested in your opinion and I think other users should have
the same requirements (querying indexed variables) - how did they solve
this?
The handling of indexes needs to be automatic. The rule of thumb is that
if you use where clause like b = , c = , a= than you need an compound index
in the same order being index(b,c,a). This is handled by the database when
it analyzes the query it tries to find an appropriate index. Or seeing it
from the other side: If you issue a query with index and the index does not
fit the query it is a problem anyway. Are you sure you have only this one
thing in the where query?
I use e.g.
COInstalledApp selectOne: {
'device.id' -> aString.
'application.__id' -> anApplication voyageId } asDictionary
on a collection that has approx. 1 Mio. entires and it is fast. I didn't
check with explain but I'll do if I find time :)
If I were you I would step in the debugger down to the Voyage part after
serialization. Then you have the exact Dictionary that is used to send to
mongo. You can then use this query to do an "explain" on it and that should
show you something. Another thing you can try (although I don't think makes
any difference) is to use the non-javascript query variant. That would be
Trip selectMany: {
'receipts' -> {
'receiptDescription' -> 'Tankquittung' } asDictionary } asDictionary
Norbert
Regards
Sabine
2015-08-04 14:47 GMT+02:00 Sabine Manaa <[hidden email]
Post by Sabine Manaa
Hi Norbert,
I didn't want to use "use database" for queries but for defining the
index from within smalltalk. But if creating indexes is not supported, I
will set the indexes within the mongo console.
Post by Sabine Manaa
The explain option is very helpful I did not know this.
Your answer was very helpful for me, thanks a lot!
Sabine
Post by Norbert Hartl
Sabine,
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with
this
Post by Sabine Manaa
Post by Norbert Hartl
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr
mongodb
Post by Sabine Manaa
Post by Norbert Hartl
console
1) how can I send mongo console commands directly from smalltalk, e.g.
use
Post by Sabine Manaa
Post by Norbert Hartl
database, set index etc.
I don't know why you want to use "use database". For queries you don't
need this because you choose the database and collection by using the
smalltalk classes. You can look there if something you need exists.
Post by Sabine Manaa
Post by Norbert Hartl
Creating indexes is not yet supported. I want to do that myself. As
always there seems to be to less people implementing stuff like that.
Post by Sabine Manaa
Post by Norbert Hartl
Is this possible?
I did not succeed with the mongo command: method, it always returns "no
such
Post by Sabine Manaa
Post by Norbert Hartl
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' }
asDictionary
Post by Sabine Manaa
Post by Norbert Hartl
should do. You can use just the nested properties form. The index will
be selected automatically. If you are not sure about queries read about
explain [1] to optimize that stuff.
Post by Sabine Manaa
Post by Norbert Hartl
[1]
http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain
Post by Sabine Manaa
Post by Norbert Hartl
Norbert
I did not succeed with the description [2] I am not sure if it is
covered by
Post by Sabine Manaa
Post by Norbert Hartl
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
Post by Sabine Manaa
Post by Norbert Hartl
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries
--
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Post by Sabine Manaa
Post by Norbert Hartl
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
________________________________
If you reply to this email, your message will be added to the
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html
Post by Sabine Manaa
Post by Norbert Hartl
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed
fields?, click here.
Post by Sabine Manaa
Post by Norbert Hartl
NAML
________________________________
View this message in context: Re: MongoDB: console commands? query
embedded indexed fields?
Post by Sabine Manaa
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
------------------------------
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841238.html
To start a new topic under Pharo Smalltalk Users, email
To unsubscribe from MongoDB: console commands? query embedded indexed
fields?, click here
<http://forum.world.st/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4840882&code=bWFuYWEuc2FiaW5lQGdtYWlsLmNvbXw0ODQwODgyfC0xOTE3OTcxOTg5>
.
NAML
<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841464.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Norbert Hartl
2015-08-07 13:09:04 UTC
Permalink
Post by Sabine Manaa
Hi Norbert, Hi Phil,
thanks for your help! Now I know what the problem is/was. :-)))
The query was right, the index was used, all is fine.
I used an inspector to look at the query's result and opening the inspector forced to load all the other objects which belong to the object found.
Explanation: I have a person which has N trips with N receipts, all with back pointers. Opening an Inspector with one trip loads its person and all other trips of this person hahaha. The magritte description isLazy but the inspector forced the loading of the objects in the VOMongoRepository at first time after image starting.
Time millisecondsToRun: [(RKATrip selectMany: { 'receipts.receiptDescription' -> 'a' } asDictionary) size] 764 .
Time millisecondsToRun: [(RKATrip selectMany: { 'receipts.receiptDescription' -> 'a' } asDictionary) inspect] 17234
So, I messed around with myself.
Well, we call this an inversed Heisenbug, don't we? :) That is a problem we should be aware when using proxy implementations and tools like gt tools. It is very likely that a powerful tool will inquire special information on a proxy that will trigger it.
Post by Sabine Manaa
Currently I do not have so much data but I hope in future there will be a lot and I want the application prepared for that.
It depends on the tool and your model. Are you storing those backreferences in mongo as well?

Norbert
Post by Sabine Manaa
BTW: I found this nice document: https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/ws/Voyage/Voyage.pdf <https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/ws/Voyage/Voyage.pdf>
I was mentioned there, thanks! Esteban, if you want, you could change my name, I married and changed my name to Manaa.
have a nice weekend
Sabine
Sabine,
Post by Sabine Manaa
Norbert,
I have to come back to your answer.
With the .explain option, I can see, how performance is with and without index on the _mongo console_.
As an example, I see: with index only 2 objects scanned, without: 200. Performance is perfect with index.
But _from smalltalk_, it seems to iterate over all objects, also with the index set. Slow.
You wrote: "The index will be selected automatically." It seems that it does not.
Usually it does.
Post by Sabine Manaa
MongoQuery has an attribute named "where" which is filled with the mentioned "receipts.receiptDescription".
http://docs.mongodb.org/manual/reference/operator/query/where/ <http://docs.mongodb.org/manual/reference/operator/query/where/>
Is this right?
In this case with the statement
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
the index would not be set automatically.
"The $where provides greater flexibility, but requires that the database processes the JavaScript expression or function for each document in the collection. "
Then my question is if the mongo classes support database queries with index and if yes, how.
I am very interested in your opinion and I think other users should have the same requirements (querying indexed variables) - how did they solve this?
The handling of indexes needs to be automatic. The rule of thumb is that if you use where clause like b = , c = , a= than you need an compound index in the same order being index(b,c,a). This is handled by the database when it analyzes the query it tries to find an appropriate index. Or seeing it from the other side: If you issue a query with index and the index does not fit the query it is a problem anyway. Are you sure you have only this one thing in the where query?
I use e.g.
COInstalledApp selectOne: {
'device.id <http://device.id/>' -> aString.
'application.__id' -> anApplication voyageId } asDictionary
on a collection that has approx. 1 Mio. entires and it is fast. I didn't check with explain but I'll do if I find time :)
If I were you I would step in the debugger down to the Voyage part after serialization. Then you have the exact Dictionary that is used to send to mongo. You can then use this query to do an "explain" on it and that should show you something. Another thing you can try (although I don't think makes any difference) is to use the non-javascript query variant. That would be
Trip selectMany: {
'receipts' -> {
'receiptDescription' -> 'Tankquittung' } asDictionary } asDictionary
Norbert
Post by Sabine Manaa
Regards
Sabine
Post by Sabine Manaa
Hi Norbert,
I didn't want to use "use database" for queries but for defining the index from within smalltalk. But if creating indexes is not supported, I will set the indexes within the mongo console.
The explain option is very helpful I did not know this.
Your answer was very helpful for me, thanks a lot!
Sabine
Post by Norbert Hartl
Sabine,
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with this
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr mongodb
console
1) how can I send mongo console commands directly from smalltalk, e.g. use
database, set index etc.
I don't know why you want to use "use database". For queries you don't need this because you choose the database and collection by using the smalltalk classes. You can look there if something you need exists.
Creating indexes is not yet supported. I want to do that myself. As always there seems to be to less people implementing stuff like that.
Is this possible?
I did not succeed with the mongo command: method, it always returns "no such
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
should do. You can use just the nested properties form. The index will be selected automatically. If you are not sure about queries read about explain [1] to optimize that stuff.
[1] http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain <http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain>
Norbert
I did not succeed with the description [2] I am not sure if it is covered by
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents <http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents>
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries <http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries>
--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html <http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html>
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com <http://nabble.com/>.
________________________________
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html <http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html>
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here.
NAML
________________________________
View this message in context: Re: MongoDB: console commands? query embedded indexed fields?
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com <http://nabble.com/>.
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841238.html <http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841238.html>
To start a new topic under Pharo Smalltalk Users, email [hidden email] <x-msg://13/user/SendEmail.jtp?type=node&node=4841464&i=1>
To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here <applewebdata://0FF8665B-B528-4730-8147-3BAABD7C79BF>.
NAML <http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
View this message in context: Re: MongoDB: console commands? query embedded indexed fields? <http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841464.html>
Sent from the Pharo Smalltalk Users mailing list archive <http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> at Nabble.com <http://nabble.com/>.
Sabine Manaa
2015-08-08 08:38:52 UTC
Permalink
Hi Norbert,

lol, I had to google that: https://en.wikipedia.org/wiki/Heisenbug
funny but that's it. It is always a good feeling when I know what the
reason for a certain behavior is.
Now I can proceed with the implementation of the search functionality.

no, in Mongo I don't store the back references, they are created when the
objects are instantiated in pharo.
Wouldn't you do it like this? What is the background for your question?

Regards
Sabine

2015-08-07 15:06 GMT+02:00 NorbertHartl [via Smalltalk] <
Am 07.08.2015 um 14:53 schrieb Sabine Manaa <[hidden email]
Hi Norbert, Hi Phil,
thanks for your help! Now I know what the problem is/was. :-)))
The query was right, the index was used, all is fine.
I used an inspector to look at the query's result and opening the
inspector forced to load all the other objects which belong to the object
found.
Explanation: I have a person which has N trips with N receipts, all with
back pointers. Opening an Inspector with one trip loads its person and all
other trips of this person hahaha. The magritte description isLazy but the
inspector forced the loading of the objects in the VOMongoRepository at
first time after image starting.
Time millisecondsToRun: [(RKATrip selectMany: {
'receipts.receiptDescription' -> 'a' } asDictionary) size] 764 .
Time millisecondsToRun: [(RKATrip selectMany: {
'receipts.receiptDescription' -> 'a' } asDictionary) inspect] 17234
So, I messed around with myself.
Well, we call this an inversed Heisenbug, don't we? :) That is a problem
we should be aware when using proxy implementations and tools like gt
tools. It is very likely that a powerful tool will inquire special
information on a proxy that will trigger it.
Currently I do not have so much data but I hope in future there will be a
lot and I want the application prepared for that.
It depends on the tool and your model. Are you storing those
backreferences in mongo as well?
Norbert
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/ws/Voyage/Voyage.pdf
I was mentioned there, thanks! Esteban, if you want, you could change my
name, I married and changed my name to Manaa.
have a nice weekend
Sabine
2015-08-06 15:36 GMT+02:00 NorbertHartl [via Smalltalk] <<a
href="x-msg://13/user/SendEmail.jtp?type=node&amp;node=4841464&amp;i=0"
Sabine,
Post by Norbert Hartl
Am 06.08.2015 um 14:49 schrieb Sabine Manaa <[hidden email]
Norbert,
I have to come back to your answer.
With the .explain option, I can see, how performance is with and without
index on the _mongo console_.
As an example, I see: with index only 2 objects scanned, without: 200.
Performance is perfect with index.
But _from smalltalk_, it seems to iterate over all objects, also with the
index set. Slow.
You wrote: "The index will be selected automatically." It seems that it does not.
Usually it does.
MongoQuery has an attribute named "where" which is filled with the
mentioned "receipts.receiptDescription".
Because of the name of this attribute "where" and the slow performance, I
http://docs.mongodb.org/manual/reference/operator/query/where/
Is this right?
In this case with the statement
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
the index would not be set automatically.
"The $where provides greater flexibility, but requires that the database
processes the JavaScript expression or function for each document in the
collection. "
Then my question is if the mongo classes support database queries with
index and if yes, how.
I am very interested in your opinion and I think other users should have
the same requirements (querying indexed variables) - how did they solve
this?
The handling of indexes needs to be automatic. The rule of thumb is that
if you use where clause like b = , c = , a= than you need an compound index
in the same order being index(b,c,a). This is handled by the database when
it analyzes the query it tries to find an appropriate index. Or seeing it
from the other side: If you issue a query with index and the index does not
fit the query it is a problem anyway. Are you sure you have only this one
thing in the where query?
I use e.g.
COInstalledApp selectOne: {
'device.id' -> aString.
'application.__id' -> anApplication voyageId } asDictionary
on a collection that has approx. 1 Mio. entires and it is fast. I didn't
check with explain but I'll do if I find time :)
If I were you I would step in the debugger down to the Voyage part after
serialization. Then you have the exact Dictionary that is used to send to
mongo. You can then use this query to do an "explain" on it and that should
show you something. Another thing you can try (although I don't think makes
any difference) is to use the non-javascript query variant. That would be
Trip selectMany: {
'receipts' -> {
'receiptDescription' -> 'Tankquittung' } asDictionary } asDictionary
Norbert
Regards
Sabine
2015-08-04 14:47 GMT+02:00 Sabine Manaa <[hidden email]
Post by Sabine Manaa
Hi Norbert,
I didn't want to use "use database" for queries but for defining the
index from within smalltalk. But if creating indexes is not supported, I
will set the indexes within the mongo console.
Post by Sabine Manaa
The explain option is very helpful I did not know this.
Your answer was very helpful for me, thanks a lot!
Sabine
2015-08-04 10:42 GMT+02:00 NorbertHartl [via Smalltalk] <[hidden
Post by Norbert Hartl
Sabine,
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts
like
Post by Sabine Manaa
Post by Norbert Hartl
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with
this
Post by Sabine Manaa
Post by Norbert Hartl
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr
mongodb
Post by Sabine Manaa
Post by Norbert Hartl
console
1) how can I send mongo console commands directly from smalltalk, e.g.
use
Post by Sabine Manaa
Post by Norbert Hartl
database, set index etc.
I don't know why you want to use "use database". For queries you don't
need this because you choose the database and collection by using the
smalltalk classes. You can look there if something you need exists.
Post by Sabine Manaa
Post by Norbert Hartl
Creating indexes is not yet supported. I want to do that myself. As
always there seems to be to less people implementing stuff like that.
Post by Sabine Manaa
Post by Norbert Hartl
Is this possible?
I did not succeed with the mongo command: method, it always returns
"no such
Post by Sabine Manaa
Post by Norbert Hartl
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' }
asDictionary
Post by Sabine Manaa
Post by Norbert Hartl
should do. You can use just the nested properties form. The index will
be selected automatically. If you are not sure about queries read about
explain [1] to optimize that stuff.
Post by Sabine Manaa
Post by Norbert Hartl
[1]
http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain
Post by Sabine Manaa
Post by Norbert Hartl
Norbert
I did not succeed with the description [2] I am not sure if it is
covered by
Post by Sabine Manaa
Post by Norbert Hartl
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
Post by Sabine Manaa
Post by Norbert Hartl
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries
--
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Post by Sabine Manaa
Post by Norbert Hartl
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com
<http://nabble.com/>.
Post by Sabine Manaa
Post by Norbert Hartl
________________________________
If you reply to this email, your message will be added to the
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html
Post by Sabine Manaa
Post by Norbert Hartl
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed
fields?, click here.
Post by Sabine Manaa
Post by Norbert Hartl
NAML
________________________________
View this message in context: Re: MongoDB: console commands? query
embedded indexed fields?
Post by Sabine Manaa
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com
<http://nabble.com/>.
------------------------------
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841238.html
To start a new topic under Pharo Smalltalk Users, email <a
href="x-msg://13/user/SendEmail.jtp?type=node&amp;node=4841464&amp;i=1"
target="_top" rel="nofollow" link="external" class="">[hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here.
NAML
<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
------------------------------
View this message in context: Re: MongoDB: console commands? query embedded indexed fields?
<http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841464.html>
Sent from the Pharo Smalltalk Users mailing list archive
<http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> at Nabble.com
<http://nabble.com/>.
------------------------------
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841471.html
To start a new topic under Pharo Smalltalk Users, email
To unsubscribe from MongoDB: console commands? query embedded indexed
fields?, click here
<http://forum.world.st/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4840882&code=bWFuYWEuc2FiaW5lQGdtYWlsLmNvbXw0ODQwODgyfC0xOTE3OTcxOTg5>
.
NAML
<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841592.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Norbert Hartl
2015-08-08 20:45:21 UTC
Permalink
Post by Sabine Manaa
Hi Norbert,
lol, I had to google that: https://en.wikipedia.org/wiki/Heisenbug
funny but that's it. It is always a good feeling when I know what the reason for a certain behavior is.
Now I can proceed with the implementation of the search functionality.
no, in Mongo I don't store the back references, they are created when the objects are instantiated in pharo.
Wouldn't you do it like this? What is the background for your question?
The background is that there is a difference between an in-memory graph and a persisted one. So storing forth and backreferences is asking for trouble. Having back references being transient and reestablishing them on materialization is a good way of having the best of both world.

Norbert
Post by Sabine Manaa
Regards
Sabine
Post by Norbert Hartl
Post by Sabine Manaa
Hi Norbert, Hi Phil,
thanks for your help! Now I know what the problem is/was. :-)))
The query was right, the index was used, all is fine.
I used an inspector to look at the query's result and opening the inspector forced to load all the other objects which belong to the object found.
Explanation: I have a person which has N trips with N receipts, all with back pointers. Opening an Inspector with one trip loads its person and all other trips of this person hahaha. The magritte description isLazy but the inspector forced the loading of the objects in the VOMongoRepository at first time after image starting.
Time millisecondsToRun: [(RKATrip selectMany: { 'receipts.receiptDescription' -> 'a' } asDictionary) size] 764 .
Time millisecondsToRun: [(RKATrip selectMany: { 'receipts.receiptDescription' -> 'a' } asDictionary) inspect] 17234
So, I messed around with myself.
Well, we call this an inversed Heisenbug, don't we? :) That is a problem we should be aware when using proxy implementations and tools like gt tools. It is very likely that a powerful tool will inquire special information on a proxy that will trigger it.
Post by Sabine Manaa
Currently I do not have so much data but I hope in future there will be a lot and I want the application prepared for that.
It depends on the tool and your model. Are you storing those backreferences in mongo as well?
Norbert
Post by Sabine Manaa
BTW: I found this nice document: https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/ws/Voyage/Voyage.pdf
I was mentioned there, thanks! Esteban, if you want, you could change my name, I married and changed my name to Manaa.
have a nice weekend
Sabine
Post by Norbert Hartl
Sabine,
Post by Sabine Manaa
Norbert,
I have to come back to your answer.
With the .explain option, I can see, how performance is with and without index on the _mongo console_.
As an example, I see: with index only 2 objects scanned, without: 200. Performance is perfect with index.
But _from smalltalk_, it seems to iterate over all objects, also with the index set. Slow.
You wrote: "The index will be selected automatically." It seems that it does not.
Usually it does.
Post by Sabine Manaa
MongoQuery has an attribute named "where" which is filled with the mentioned "receipts.receiptDescription".
http://docs.mongodb.org/manual/reference/operator/query/where/
Is this right?
In this case with the statement
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
the index would not be set automatically.
"The $where provides greater flexibility, but requires that the database processes the JavaScript expression or function for each document in the collection. "
Then my question is if the mongo classes support database queries with index and if yes, how.
I am very interested in your opinion and I think other users should have the same requirements (querying indexed variables) - how did they solve this?
The handling of indexes needs to be automatic. The rule of thumb is that if you use where clause like b = , c = , a= than you need an compound index in the same order being index(b,c,a). This is handled by the database when it analyzes the query it tries to find an appropriate index. Or seeing it from the other side: If you issue a query with index and the index does not fit the query it is a problem anyway. Are you sure you have only this one thing in the where query?
I use e.g.
COInstalledApp selectOne: {
'device.id' -> aString.
'application.__id' -> anApplication voyageId } asDictionary
on a collection that has approx. 1 Mio. entires and it is fast. I didn't check with explain but I'll do if I find time :)
If I were you I would step in the debugger down to the Voyage part after serialization. Then you have the exact Dictionary that is used to send to mongo. You can then use this query to do an "explain" on it and that should show you something. Another thing you can try (although I don't think makes any difference) is to use the non-javascript query variant. That would be
Trip selectMany: {
'receipts' -> {
'receiptDescription' -> 'Tankquittung' } asDictionary } asDictionary
Norbert
Post by Sabine Manaa
Regards
Sabine
Post by Sabine Manaa
Hi Norbert,
I didn't want to use "use database" for queries but for defining the index from within smalltalk. But if creating indexes is not supported, I will set the indexes within the mongo console.
The explain option is very helpful I did not know this.
Your answer was very helpful for me, thanks a lot!
Sabine
Post by Norbert Hartl
Sabine,
Hi,
1) how can I send mongo console commands directly from smalltalk
2) how to query on an embedded collection with an index
I have trips and each trip has an embedded collection of N receipts like
{ "tripName" : "trip 1",
{"receipts" : [
{"receiptDescription" : "receipt 1" }
{"receiptDescription" : "receipt 2" }
{"receiptDescription" : "receipt 3" } ],
}
For quick search, I created an index on the embeded[1] receipts with this
command at the mongoDB console in a terminal (NOT in smalltalk)
db.Trips.createIndex({"receipts.receiptDescription":1})
with a command like this in the mongoDB console
db.Trips.find({"receipts.receiptDescription":"receipt 1"})
I get the resulting receipt.
Before, I have to set the database with "use {databasename}" in thr mongodb
console
1) how can I send mongo console commands directly from smalltalk, e.g. use
database, set index etc.
I don't know why you want to use "use database". For queries you don't need this because you choose the database and collection by using the smalltalk classes. You can look there if something you need exists.
Creating indexes is not yet supported. I want to do that myself. As always there seems to be to less people implementing stuff like that.
Is this possible?
I did not succeed with the mongo command: method, it always returns "no such
cmd".
2) how can I search for a receipt which is indexed as described from
smalltalk?
A simple
Trip selectMany: { 'receipts.receiptDescription' -> 'Tankquittung' } asDictionary
should do. You can use just the nested properties form. The index will be selected automatically. If you are not sure about queries read about explain [1] to optimize that stuff.
[1] http://docs.mongodb.org/manual/reference/method/cursor.explain/#cursor.explain
Norbert
I did not succeed with the description [2] I am not sure if it is covered by
it.
Regards
Sabine
[1]
http://docs.mongodb.org/manual/core/index-multikey/#index-arrays-with-embedded-documents
[2] http://smallworks.eu/web/blog/2013-07-18-Voyage-advanced-queries
--
View this message in context: http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
________________________________
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4840888.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here.
NAML
________________________________
View this message in context: Re: MongoDB: console commands? query embedded indexed fields?
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841238.html
To start a new topic under Pharo Smalltalk Users, email <a href="x-msg://13/user/SendEmail.jtp?type=node&amp;node=4841464&amp;i=1" target="_top" rel="nofollow" link="external" class="">[hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here.
NAML
View this message in context: Re: MongoDB: console commands? query embedded indexed fields?
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
http://forum.world.st/MongoDB-console-commands-query-embedded-indexed-fields-tp4840882p4841471.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from MongoDB: console commands? query embedded indexed fields?, click here.
NAML
View this message in context: Re: MongoDB: console commands? query embedded indexed fields?
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Loading...