[xwiki-devs] integrating media streaming with xwiki? (was Re: Problems with scripting)
but do you think it's possible to implement a streaming service in xwiki using groovy class with importing a streaming api(the streaming server will be vlc for example)?
Xwiki is better suited as a framework for creating and presenting streaming applications but not for the actual streaming implementation. It is correct to look at implementations specifically designed for streaming (e.g. VLC, Darwin <http://developer.apple.com/opensource/server/streaming/index.html>, etc) rather than trying to tame performance issues such as object/thread overhead,. out-of-memory and running out of threads -- common issues when many long-running streaming downloads occur in a servlet environment.. On the client end, Xwiki creates the HTML/CSS/javascript scaffolding, with streaming implemented via streaming "plugins" in the web-browser (e.g. mozilla plugin, a java applet or flash). These plugins could be accessed in Xwiki via special velocity macros that cleanly encapsulate all the scaffolding required to achieve this. The macro would allow easy parametrization and placement of these "streaming UI objects" in a Xwiki-based document/application. For example VLC has a mozilla broser plugin that you can control with a javascript API. http://www.videolan.org/doc/play-howto/en/ch04.html#id310965 suggests the following could be encapsulated within a velocity macro to implement this functionality: <embed type="application/x-vlc-plugin" pluginspage="http://www.videolan.org" version="VideoLAN.VLCPlugin.2" width="640" height="480" id="vlc"> </embed> <script language="Javascript"> <!-- var vlc = document.getElementById("vlc"); vlc.audio.toggleMute(); !--> </script> In my experience, it is straightforward to integrate these kinds of things with Xwiki, once you get adjusted to the possiblity of "computation" happening in three places and at different times: browser, server and plugin; and figuring out the dependencies and interactions therein. Unfortunately, from a pragmatic perspective, expecting your users to install VLC or other special browser plugins is the kiss-of-death. Most users will just blow it off and move on to something else that uses Flash and "works right out of the box" for the majority of users. (Therefore you can be doctrinaire about being "purely open source" and it just means you'll be strongly limiting your customer base and needlessly limiting your own project's potential for success). It would be nice to have a truly abstract streaming media macro that would implement the same generic API for streaming media within Xwiki. The macro would determine at run-time, whether the browser is enabled with specific plugins needed to display a specific media type (such as the VLC moz plugin); if not available, the macro would dynamically switch-in the flash-based equivalent for people using IE or who don't want to install the VLC moz plugin. For example given a "mp4", the macro would first attempt to use VLC, on failure it might try the platform's browser-embeddable media presenter if media-compatible (e.g. MS windows media), and would finally try a "generic" flash player such as the Jeroen "JW" Wijering players ( http://www.longtailvideo.com/players/ ). Who else is working on streaming media issues in Xwiki? Would love to see some group interest and coding participation towards better integration of streaming media in Xwiki. Niels http://nielsmayer.com PS: some of my work-in-progress on some streaming clients in Xwiki (not actual velocity macros yet, but at least they work). http://nielsmayer.com/xwiki/bin/view/Macros/JW-Player?raw=1 http://nielsmayer.com/xwiki/bin/view/Macros/YT-Chromeless?raw=1
Hi I was thinking of a usecase where users can play an attachment for an example. I'm pretty new to streaming content, but to enable streaming of attachments (audio, video) we would have to store them on files instead in database entries right? and then may be we can use JMF ( http://java.sun.com/products/java-media/jmf/index.jsp) for streaming. Thanks. - Asiri
On Sat, Apr 11, 2009 at 10:20 PM, Asiri Rathnayake < [email protected]> wrote:
I was thinking of a usecase where users can play an attachment for an example.
I'm pretty new to streaming content, but to enable streaming of attachments (audio, video) we would have to store them on files instead in database entries right? and then may be we can use JMF ( http://java.sun.com/products/java-media/jmf/index.jsp) for streaming.
If you're playing back an Xwiki attachment as streaming media, there's plenty of problems because the entire stack is not designed for streaming... On the server-end, things are limited not by Xwiki itself, but by the use of Java and Servlets, which ends up being incredibly heavy-weight for streaming large data files. The issue with large-files and streaming media (e.g. video) is that the "download" happens over a very long period of time; streaming playback by definition means you're not downloading entire files before play-back; instead, you use a small buffer and hope that network can keep the buffer filled while the data plays back at a slow "real-time" data-rate. Even if xwiki's db access could be made "incremental" or "paged", with multiple users streaming simultaneously, a java/servlet based system requires huge amounts of overhead per connection. It's easy to run out of memory, or max-threads. The worst part about this architecture, as opposed to more traditional practices in C/C++ using select()<http://en.wikipedia.org/wiki/Select_%28Unix%29>and nonblocking I/O<http://en.wikipedia.org/wiki/Non-blocking_I/O#Select.28.2Fpoll.29_loops>, is that most of those threads are basically doing *nothing* most of the time, and when they are, they are sending out packet of data and then going back to sleep... Of course, the servlet spec allows streaming, and there are implementations of servlet-streaming that are quite straightforward to implement, but may have performance/overhead issues anyways in real-use, e.g.: http://www.longtailvideo.com/support/forum/Modules/14410/Servlet-Streaming IMHO, the leading implementations are still in C/C++ -- just find one that's active, mature, and widely used, treat it as a black box, and use whatever combination of arcane configuration commands needed to get it integrated:* *The most interesting option for Xwiki integration is mod_h264_streaming<http://h264.code-shop.com/trac/>since it would run in Apache and many already "front" their Xwiki with apache and mod_jk. (This option might provide and easier authentication&access-control integration w/ Xwiki). Some other standalone implementations include: Darwin Streaming Server by Apple<http://developer.apple.com/darwin/projects/streaming/>, ViTooKi <http://vitooki.sourceforge.net/>, Catra<http://www.catrasoftware.it/Streaming/CatraStreamingPlatform.htm>. Those might do the job better for managing a site full of streaming servers, but may be harder to integrate. I haven't looked at JMF solution for streaming.... nor do I see anything about it that would make the task easier or the issues I mentioned go away. Please let us know what you find out regarding JMF's performance and integration capabilities! Niels http://nielsmayer.com
Maybe I am hammering he pleasure of contradiction but streaming in java servlets definitely is possible and done. The best example I know of is red5: http://osflash.org/red5 which is the backbone, among others, of DimDim, one of the teleconferencing software. In general, however, I am rather a fan of Darwin Streaming Server, open-source C++, but I have almost done no programming with it. The best documentation I found for it is at soundscreen.com. Storage at xwiki side definitely needs a fix before videos can be made attachment but that doesn't seem like an impossible mission. Re- encoding for streaming maybe is the hardest. paul Le 14-avr.-09 à 22:18, Niels Mayer a écrit :
On Sat, Apr 11, 2009 at 10:20 PM, Asiri Rathnayake < [email protected]> wrote:
I was thinking of a usecase where users can play an attachment for an example.
I'm pretty new to streaming content, but to enable streaming of attachments (audio, video) we would have to store them on files instead in database entries right? and then may be we can use JMF ( http://java.sun.com/products/java-media/jmf/index.jsp) for streaming.
If you're playing back an Xwiki attachment as streaming media, there's plenty of problems because the entire stack is not designed for streaming...
On the server-end, things are limited not by Xwiki itself, but by the use of Java and Servlets, which ends up being incredibly heavy-weight for streaming large data files. The issue with large-files and streaming media (e.g. video) is that the "download" happens over a very long period of time; streaming playback by definition means you're not downloading entire files before play-back; instead, you use a small buffer and hope that network can keep the buffer filled while the data plays back at a slow "real-time" data-rate.
Even if xwiki's db access could be made "incremental" or "paged", with multiple users streaming simultaneously, a java/servlet based system requires huge amounts of overhead per connection. It's easy to run out of memory, or max-threads. The worst part about this architecture, as opposed to more traditional practices in C/C++ using select()<http://en.wikipedia.org/wiki/Select_%28Unix%29>and nonblocking I/O<http://en.wikipedia.org/wiki/Non-blocking_I/O#Select.28.2Fpoll.29_loops
, is that most of those threads are basically doing *nothing* most of the time, and when they are, they are sending out packet of data and then going back to sleep... Of course, the servlet spec allows streaming, and there are implementations of servlet-streaming that are quite straightforward to implement, but may have performance/overhead issues anyways in real- use, e.g.: http://www.longtailvideo.com/support/forum/Modules/14410/Servlet-Streaming
IMHO, the leading implementations are still in C/C++ -- just find one that's active, mature, and widely used, treat it as a black box, and use whatever combination of arcane configuration commands needed to get it integrated:* *The most interesting option for Xwiki integration is mod_h264_streaming<http://h264.code-shop.com/trac/>since it would run in Apache and many already "front" their Xwiki with apache and mod_jk. (This option might provide and easier authentication&access-control integration w/ Xwiki). Some other standalone implementations include: Darwin Streaming Server by Apple<http://developer.apple.com/darwin/projects/streaming/>, ViTooKi <http://vitooki.sourceforge.net/>, Catra<http://www.catrasoftware.it/Streaming/ CatraStreamingPlatform.htm>. Those might do the job better for managing a site full of streaming servers, but may be harder to integrate.
I haven't looked at JMF solution for streaming.... nor do I see anything about it that would make the task easier or the issues I mentioned go away. Please let us know what you find out regarding JMF's performance and integration capabilities!
Niels http://nielsmayer.com _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
Niels Mayer wrote:
On Sat, Apr 11, 2009 at 10:20 PM, Asiri Rathnayake < [email protected]> wrote:
I was thinking of a usecase where users can play an attachment for an example.
I'm pretty new to streaming content, but to enable streaming of attachments (audio, video) we would have to store them on files instead in database entries right? and then may be we can use JMF ( http://java.sun.com/products/java-media/jmf/index.jsp) for streaming.
If you're playing back an Xwiki attachment as streaming media, there's plenty of problems because the entire stack is not designed for streaming...
On the server-end, things are limited not by Xwiki itself, but by the use of Java and Servlets, which ends up being incredibly heavy-weight for streaming large data files. The issue with large-files and streaming media (e.g. video) is that the "download" happens over a very long period of time; streaming playback by definition means you're not downloading entire files before play-back; instead, you use a small buffer and hope that network can keep the buffer filled while the data plays back at a slow "real-time" data-rate.
Even if xwiki's db access could be made "incremental" or "paged", with multiple users streaming simultaneously, a java/servlet based system requires huge amounts of overhead per connection. It's easy to run out of memory, or max-threads. The worst part about this architecture, as opposed to more traditional practices in C/C++ using select()<http://en.wikipedia.org/wiki/Select_%28Unix%29>and nonblocking I/O<http://en.wikipedia.org/wiki/Non-blocking_I/O#Select.28.2Fpoll.29_loops>, is that most of those threads are basically doing *nothing* most of the time, and when they are, they are sending out packet of data and then going back to sleep... Of course, the servlet spec allows streaming, and there are implementations of servlet-streaming that are quite straightforward to implement, but may have performance/overhead issues anyways in real-use, e.g.: http://www.longtailvideo.com/support/forum/Modules/14410/Servlet-Streaming
Modern containers have support for NIO and long-lasting requests (cometd). It's just a matter of configuration.
IMHO, the leading implementations are still in C/C++ -- just find one that's active, mature, and widely used, treat it as a black box, and use whatever combination of arcane configuration commands needed to get it integrated:* *The most interesting option for Xwiki integration is mod_h264_streaming<http://h264.code-shop.com/trac/>since it would run in Apache and many already "front" their Xwiki with apache and mod_jk. (This option might provide and easier authentication&access-control integration w/ Xwiki). Some other standalone implementations include: Darwin Streaming Server by Apple<http://developer.apple.com/darwin/projects/streaming/>, ViTooKi <http://vitooki.sourceforge.net/>, Catra<http://www.catrasoftware.it/Streaming/CatraStreamingPlatform.htm>. Those might do the job better for managing a site full of streaming servers, but may be harder to integrate.
I haven't looked at JMF solution for streaming.... nor do I see anything about it that would make the task easier or the issues I mentioned go away. Please let us know what you find out regarding JMF's performance and integration capabilities!
JMF would just help on the streaming side, but there are many other tools that can do that. The hard part is the database, and the way we handle attachments. -- Sergiu Dumitriu http://purl.org/net/sergiu/
On Wed, Apr 15, 2009 at 10:34 AM, Sergiu Dumitriu <[email protected]> wrote:
Niels Mayer wrote: ...
Of course, the servlet spec allows streaming, and there are implementations of servlet-streaming that are quite straightforward to implement, but may have performance/overhead issues anyways in real-use,
... http://www.longtailvideo.com/support/forum/Modules/14410/Servlet-Streaming
Modern containers have support for NIO and long-lasting requests (cometd). It's just a matter of configuration.
Yes, I gave the above example to show how streaming service can be achieved in modern container. I think this is classified as a religious debate. In looking for something to backup my point, I found this article suggesting the threads vs events performance payoff is only 5-10%: ( from http://www.cs.toronto.edu/syslab/courses/csc2231/05au/lectures/lecture04.pdf) Concurrency management • A religious topic: threads vs. events – Threads • Easier to program • Easy to understand and exploit parallelism (multi-proc) – Events • Easier to program • Scheduling can be controlled and exploited – Not hidden in the thread scheduler or lock • Performance, scaling • All this makes sense only… – If the bottleneck is due to threads/events (unlikely) Pipeline servers: L1/L2 cache • Claim: instructions-per-cycle is low on servers – Threads hurt l-cache performance – Idea: re-architect software into computational stages • Execute each task repetitively in a stage • Problems: – Quite a drastic change in architecture – Working set size of stage must align well with l-cache size – Performance pay-off is minimal • 5-10% improvement (1 month of Moore’s law) However, that doesn't include other issues of overhead that are magnified by java and threads. And on a Unix server, it all ends up getting implemented as a select() somewhere inside java anyways. Java just ends up being a high-overhead wrapper to the bare-metal of the I/O mechanism on a server :-) This, however, is clearly a "religious" POV. Just consider the "object" and "thread" overhead needed to support 100, or 500 or 1000 concurrent long-downloads of streaming-media in java. Those are not unheard of numbers for streaming media, even small-scale. The issues are many when it comes to scaling: cache-nonlocality caused by sporadic bits of code getting executed all-over the place in a giant memory image, the unpredictable-time-response caused by garbage collections (or the additional performance penalty of incremental gc), the cache-nonlocality caused by garbage collections, etc.
JMF would just help on the streaming side, but there are many other tools that can do that. The hard part is the database, and the way we handle attachments.
Assuming this is even what you want to be doing with Xwiki. I think Xwiki should continue to do what it is doing and do it best. I think a separate project with a separate architecture that already does its job-best should be integrated w/ Xwiki to provide streaming service. Independently, it would be great to improve Xwiki's attachment capabilities so that in the future, they might prove worthy of using for streaming media out of the database.Incremental uploading and the ability to actually store
100mb attachments would be a good start....
And streaming out of the database certainly *is* an interesting thing to do: for example, being able to have an index of segments of video clips that one could stream out of a database continuously and without interruption/glitching between segments. Consider, however, that placing large media datafiles directly into the database is problematic as it can run up against system limits as well as practicality limits for administration, backup, etc. Making use of existing work on media servers where these issues have been solved and considered is a good idea. -- Niels http://nielsmayer.com PS: another interesting view on the religious topic of threads/vs/events for throughput www.cs.cornell.edu/courses/cs614/2004sp/slides/CS614%20-%20concurrency.ppt
From Concurrency, Threads, and Events by Ken Birman (Based on a slide set prepared by Robbert van Renesse)
Events-based systems use fewer resources Better performance (particularly scalability) Event-based systems harder to program Have to avoid blocking at all cost Block-structured programming doesn’t work How to do exception handling? In both cases, tuning is difficult
In practice, many kinds of systems need to support both threads and events Threaded programs in Unix are the common example of these, because window systems use events The programmer uses cthreads or pthreads Major problem: the UNIX kernel interface wasn’t designed with threads in mind!
participants (4)
-
Asiri Rathnayake -
Niels Mayer -
Paul Libbrecht -
Sergiu Dumitriu