(resending since I got a failure from xwiki's mail content filter )
---------- Forwarded message ----------
From: Vincent Massol <vincent(a)massol.net>
Date: Sun, Jul 26, 2009 at 10:21 AM
Subject: Re: [xwiki-devs] adding a new jsr-223 scripting language to
xwiki (clojure)
To: XWiki Developers <devs(a)xwiki.org>
Hi Niels,
On Jul 26, 2009, at 10:13 AM, Niels Mayer wrote:
In
http://code.xwiki.org/xwiki/bin/view/Snippets/DisplayFormatedCodeAttachment…
VincentMassol | 2009/05/05 20:03
Niels, no there's no plan but we can now
support any language using macros
so feel free to provide a clojure macro if you're interested. Shouldn't be
too hard to do, especially if there's a JSR-223 implementation (in which
case simply dropping the jar in WEB-INF/lib should be enough - you'd then
use it using the script macro).
Clojure <http://groups.google.com/group/clojure> has aJSR-223 implementation
according to:
http://github.com/pmf/clojure-jsr223/tree/master
http://groovy.codehaus.org/JSR-223+access+to+other+JVM+languages
http://sayspy.blogspot.com/2009/03/interacting-between-jvm-lang-here-and.ht…
Are there any examples, documentation, or suggestions of writing a "script
macro" to call a new jar in WEB-INF/lib ?
Well all you need is drop the JSR223 jar in WEB-INF/lib and then you
can use the new language using the script macro:
http://code.xwiki.org/xwiki/bin/view/Macros/ScriptMacro
Then if you want you can write a wrapping macro such as we've done for
groovy and jruby so that it can be called with {{clojure}} instead of
{{script language="clojure"}}.
Any other special magic I should know? Does anything
from the above suggest
that Clojure couldn't be used as an Xwiki scripting language, replacing
cases where Groovy scripting might normally be employed?
Niels
http://nielsmayer.com
PS: Here's examples of easy "scripted java" programming you can do in
Clojure (note the helpful parallelism constructs):
http://travis-whitton.blogspot.com/2009/07/network-sweeping-with-clojure.ht…
http://travis-whitton.blogspot.com/2009/06/clojures-agents-scientists-monke…
It could be very useful to employ massive parallelism via such Clojure
scripts, which could achieve a xwiki-based web portal performance akin to
Yahoo's, Google's, etc. For example, the following describes how Yahoo works
-- and would be quite easy to implement this kind of processing "for free"
in Clojure with very little code:
http://research.yahoo.com/files/pnuts.pdf
I'm definitely +1 to make it part of the XWiki platform (same as for
jruby) if you develop it. Similar to jruby I don't think we should
package it in the default WAR though but as an optional download to
install (will be even easier with the app manager).
Thanks
-Vincent
The component responsible for multi-record requests is called the
>
> scatter-gather engine, and is a component of the router.
> The scatter-gather engine receives a multi-record request,
> splits it into multiple individual requests for single records
> or single tablet scans, and initiates those requests in parallel.
> As the requests return success or failure, the scatter-gather
> engine assembles the results and then passes them to the
> client. In our implementation, the engine can begin streaming
> some results back to the client as soon as they appear.
> We chose a server-side approach instead of having the client
> initiate multiple parallel requests for several reasons. First,
> at the TCP/IP layer, it is preferable to have one connection
> per client to the PNUTS service; since there are many clients
> (and many concurrent processes per client machine) opening
> one connection to PNUTS for each record being requested
> in parallel overloads the network stack. Second, placing this
> functionality on the server side allows us to optimize, for
> example by grouping multiple requests to the same storage
> server in the same web service call.
>
> Range queries and table scans are also handled by the
> scatter gather engine. Typically there is only a single client
> process retrieving the results for a query. The scatter gather
> engine will scan only one tablet at a time and return results
> to the client; this is about as fast as a typical client can
> process results. In the case of a range scan, this mecha-
> nism simplifies the process of returning the top-K results
> (a frequently requested feature), since we only need to scan
> enough tablets to provide K results. After returning the
> first set of results, the scatter-gather engine constructs and
> returns a continuation object, which allows the client to re-
> trieve the next set of results. The continuation object con-
> tains a modified range query, which, when executed, restarts
> the range scan at the point the previous results left off. Con-
> tinuation objects allow us to have cursor state on the client
> side rather than the server. In a shared service such as
> PNUTS, it is essential to minimize the amount of server-
> side state we have to manage on behalf of clients.