Does Solr fit my needs?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Does Solr fit my needs?

G.Long
Hi there :)

I'm looking for a way to save xml files into some sort of database and
i'm wondering if Solr would fit my needs.
The xml files I want to save have a lot of child nodes which also
contain child nodes with multiple values. The depth level can be more
than 10.

After having indexed the files, I would like to be able to query for
subparts of those xml files and be able to reconstruct them as xml files
with all their children included. However, I'm wondering if it is
possible with an index like solr lucene to keep or easily recover the
structure of my xml data?

Thanks for your help,

Regards,

Gary
Reply | Threaded
Open this post in threaded view
|

RE: Does Solr fit my needs?

Robert Sandiford
Without speaking directly to the indexing and searching of the specific fields, it is certainly possible to retrieve the xml file.  While Solr isn't a DB, it does allow a binary field to be associated with an index document.  We store a GZipped XML file in a binary field and retrieve that under certain conditions to get at original document information.  We've found that Solr can handle these much faster than our DB can do.  (We regularly index a large portion of our documents, and the XML files are prone to frequent changes).  If you DO keep such a blob in your Solr index, make sure you retrieve that field ONLY when you really want it...

Now - if your XML files are relatively static (i.e. only change rarely, or only have new ones) then it still might make sense to use a real DB to store those, and just keep the primary key to the DB row in the Solr index.

Bob Sandiford | Lead Software Engineer | SirsiDynix
P: 800.288.8020 X6943 | [hidden email]
www.sirsidynix.com

Register for the 2012 COSUGI User Group Conference today for early bird pricing!
May 2-5 at Disney's Coronado Springs Resort - Lake Buena Vista, Florida
 
Join the conversation: Like us on Facebook! Follow us on Twitter!


-----Original Message-----
From: G.Long [mailto:[hidden email]]
Sent: Friday, April 27, 2012 10:32 AM
To: [hidden email]
Subject: Does Solr fit my needs?

Hi there :)

I'm looking for a way to save xml files into some sort of database and i'm wondering if Solr would fit my needs.
The xml files I want to save have a lot of child nodes which also contain child nodes with multiple values. The depth level can be more than 10.

After having indexed the files, I would like to be able to query for subparts of those xml files and be able to reconstruct them as xml files with all their children included. However, I'm wondering if it is possible with an index like solr lucene to keep or easily recover the structure of my xml data?

Thanks for your help,

Regards,

Gary


Reply | Threaded
Open this post in threaded view
|

Re: Does Solr fit my needs?

Shashi Kant-2
We have used both Solr and graph databases for our XML file indexing. Both
are equivalent in terms of performance, but a graph db (such as Neo4j)
offers a lot more flexibility in joining across the nodes and traversing.
If your data is strictly hierarchical Solr might do it, alternately suggest
looking at a Graph database such as Neo4j.



On Fri, Apr 27, 2012 at 10:36 AM, Bob Sandiford <
[hidden email]> wrote:

> ndexing and searching of the specific fields, it is certainly possible to
> retrieve the xml file.  While Solr isn't a DB, it does allow a binary field
> to be associated with an index document.  We store a GZipped XML file in a
> binary field and retrieve that under certain conditions to get at original
> document information.  We've found that Solr can handle these much faster
> than our DB can do.  (We regularly index a large portion of our documents,
> and the XML files are prone to frequent changes).  If you DO keep such a
> blob in your Solr index, make sure you retrieve that field ONLY when you
> really want it...
Reply | Threaded
Open this post in threaded view
|

Re: Does Solr fit my needs?

Walter Underwood
In reply to this post by G.Long
Solr will not keep the structure of your XML data. Solr and Lucene have a flat data model. You can map hierarchy into that, but it can be a lot of work.

I recommend starting with a dedicated XML database. MarkLogic is commercial, but they have added a free developer license that can be used for some production (see limitations here: http://community.marklogic.com/express).

I would say to use "an XML database", but there really is no other production-ready XML database. eXist is nice, but it can be 100X slower than MarkLogic.

I used to work at MarkLogic and I've implemented Solr at two major web companies. If you really want to work in XML, you'll be a lot happier with MarkLogic.

wunder
 
On Apr 27, 2012, at 7:31 AM, G.Long wrote:

> Hi there :)
>
> I'm looking for a way to save xml files into some sort of database and i'm wondering if Solr would fit my needs.
> The xml files I want to save have a lot of child nodes which also contain child nodes with multiple values. The depth level can be more than 10.
>
> After having indexed the files, I would like to be able to query for subparts of those xml files and be able to reconstruct them as xml files with all their children included. However, I'm wondering if it is possible with an index like solr lucene to keep or easily recover the structure of my xml data?
>
> Thanks for your help,
>
> Regards,
>
> Gary





Reply | Threaded
Open this post in threaded view
|

Re: Does Solr fit my needs?

Billnbell
In reply to this post by G.Long
You could use SQL Server and External Fields in Solr to get what you need from the database on result of the query.

Bill Bell
Sent from mobile


On Apr 27, 2012, at 8:31 AM, "G.Long" <[hidden email]> wrote:

> Hi there :)
>
> I'm looking for a way to save xml files into some sort of database and i'm wondering if Solr would fit my needs.
> The xml files I want to save have a lot of child nodes which also contain child nodes with multiple values. The depth level can be more than 10.
>
> After having indexed the files, I would like to be able to query for subparts of those xml files and be able to reconstruct them as xml files with all their children included. However, I'm wondering if it is possible with an index like solr lucene to keep or easily recover the structure of my xml data?
>
> Thanks for your help,
>
> Regards,
>
> Gary
Reply | Threaded
Open this post in threaded view
|

Re: Does Solr fit my needs?

G.Long
In reply to this post by G.Long
Hi :)

Thank you all for your answers. I'll try these solutions :)

Kind regards,

Gary

Le 27/04/2012 16:31, G.Long a écrit :

> Hi there :)
>
> I'm looking for a way to save xml files into some sort of database and
> i'm wondering if Solr would fit my needs.
> The xml files I want to save have a lot of child nodes which also
> contain child nodes with multiple values. The depth level can be more
> than 10.
>
> After having indexed the files, I would like to be able to query for
> subparts of those xml files and be able to reconstruct them as xml
> files with all their children included. However, I'm wondering if it
> is possible with an index like solr lucene to keep or easily recover
> the structure of my xml data?
>
> Thanks for your help,
>
> Regards,
>
> Gary

Reply | Threaded
Open this post in threaded view
|

Re: Does Solr fit my needs?

Mikhail Khludnev
no problem - you are welcome.
Nothing out-of-the-box yet. Only approach is ready

http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
https://issues.apache.org/jira/browse/SOLR-3076

Regards

On Mon, Apr 30, 2012 at 12:06 PM, G.Long <[hidden email]> wrote:

> Hi :)
>
> Thank you all for your answers. I'll try these solutions :)
>
> Kind regards,
>
> Gary
>
> Le 27/04/2012 16:31, G.Long a écrit :
>
>> Hi there :)
>>
>> I'm looking for a way to save xml files into some sort of database and
>> i'm wondering if Solr would fit my needs.
>> The xml files I want to save have a lot of child nodes which also contain
>> child nodes with multiple values. The depth level can be more than 10.
>>
>> After having indexed the files, I would like to be able to query for
>> subparts of those xml files and be able to reconstruct them as xml files
>> with all their children included. However, I'm wondering if it is possible
>> with an index like solr lucene to keep or easily recover the structure of
>> my xml data?
>>
>> Thanks for your help,
>>
>> Regards,
>>
>> Gary
>>
>
>


--
Sincerely yours
Mikhail Khludnev
Tech Lead
Grid Dynamics

<http://www.griddynamics.com>
 <[hidden email]>