23 Aug
2014
23 Aug
'14
12:18 a.m.
you are alright Paul!
One of my queries was hogging the DB. I will improve its logic... I will
check the Solr api. Considering the performance terms, the SOLR works
better than hibernate?
Thanks Paul, you are the best.
2014-08-22 18:11 GMT-03:00 Paul Libbrecht <paul(a)hoplahup.net>et>:
Danilo,
have you checked the MySQL process list?
I'd suspect something is hogging.
For search, I'd recommend to leverage solr… but with an amount of
customizations. There are some hooks in the solr-plugin, I believe.
hope it helps.
paul
On 22 août 2014, at 22:54, Danilo Oliveira <daniloa.oliveira(a)gmail.com>
wrote:
--
Danilo Amaral de Oliveira
Engenheiro de Computação
celular (32) 9111 - 6867
Hello Devs,
I am developing an application based on XWiki that is mapping,
connecting,
relating and graphical disposing movie
information in order to make
possible to the user explore their trailers.
At the beginning with a light data set (<5k movies) the application was
running well, but today I started to populate my database (MYSQL) and the
application became unusable, the queries is taking more than 5 minutes to
complete. Actually, it has more than 15k movies (1 movie = 1 doc) and I
need to upload more 100k.
I already have checked the cache and performance page but I don't know if
they[1][2] solve my problem:
I think that is a architecture challenge.
My AS IS process is:
-User insert a movie,
-the application search for the movie and their related films based on
its
characteristics (a lot of joins and other
algorithms) (bottleneck)
-the application returns the results as a map;
I am wondering if I could use the custom mapping[3] to solve my problem
due
the fact that the relationship information for
each movie, in this first
moment, don't need to change often. Each movie has X movies related,
sorted
by similarity. So, I could create some
relationship algorithm that will
run
scheduled ( 1 time by week) and populate this new
table .I am thinking to
use dataframe panda of python to talk directlly with mysql and make data
analysis, any other suggestion?
So I would create a custom map to my relationship movie class, run the
algorithm, populate the new table, so my TO BE would be:
TO BE
-user insert movie info;
-simple select on the customtable "MoviesRelated";
-the application returns the results;
I would appreciate some opinion. Thank you very much.
[1]http://platform.xwiki.org/xwiki/bin/view/AdminGuide/Performances
[2]http://extensions.xwiki.org/xwiki/bin/view/Extension/Cache+Module
[3]http://platform.xwiki.org/xwiki/bin/view/DevGuide/CustomMapping
Danilo
--
Danilo Amaral de Oliveira
Engenheiro de Computação
celular (32) 9111 - 6867
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
devs(a)xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs