OMwiki:Tech

(Difference between revisions)
(Dreamlist: link to Elphel products + wording clarifications)
(Known issues: the VP-501xl has fixed firmware, but there are higher-end models that can switch aspect ratios)
 
(23 intermediate revisions not shown)
Line 1: Line 1:
__NOTOC__
__NOTOC__
-
=='''Known issues:'''==
 
-
(Listed in order of severity.)
 
-
*Find a way to add <tt>style="width:352px;"</tt> to embed code in 352×240 streams.
+
==Recording devices==
 +
===Known issues===
 +
*Fixup non-compliment DV hardware + firewire-core [https://bugzilla.redhat.com/show_bug.cgi?id=449252 Example ticket]
-
*Video playback might take ~6 seconds to start for timecodes towards the end of the meeting (pending seeking support in [http://www.xiph.org/oggz/ oggz-chop]). Also, Google Chrome won't start playback if oggz-chop is used.
+
===Possibly fixable===
 +
*Rework [http://modcopy.sourceforge.net/ modcopy] to avoid mmap calls on JFFS2 filesystems (''e.g.'', the OLPC-XO). [http://sourceforge.net/apps/trac/modcopy/ticket/9 Ticket]
 +
*Investigate audio ground loops and prevention techniques
-
*Edit [[MediaWiki:Common.js]] to download animated gifs only [[OMwiki:Finding_aid|when 'show' is clicked]] to avoid long page loads.
+
===Dreamlist===
 +
*Find a portable camera or device that records footage from ''two'' audio sources (''e.g.'', XRL mic + ambient mic) time-synchronized to a video track.
 +
*Migrate to open-firmware cameras that record directly to Ogg; also include simplified on-camera editing.  Elphel has completely-open [http://www3.elphel.com/353_turnkey network cameras] supported by the [http://cinema.elphel.com/ Apertus] community.
 +
*Auto-balancing, wireless, golfball-sized mics that are mutable by the camera.  Best for when push-to-talk mics are not available.
 +
*Have cameras set to blur-out individuals broadcasting a 'do not record my voice, person, or both' flag ''via'' iPhones, RFID, or colored stickers as mentioned in Yochai Benkler's keynote at OVC 2009.
-
*ffmpeg on Internet Archive servers is ver. 0.5, not recent enough to generate/display thumbnails.  Animated thumbnails are now self-generated; static thumbnails are skipped due to the additional work involved.
+
<br>
-
*[[OMwiki:Speakers|Speaker labels]] work for annotations but not transcripts.  Also need to figure out AJAX auto-complete magic.
+
==Editing==
 +
===Known issues===
 +
*[http://cinelerra.org/ Cinelerra-CV] is buggy and somewhat dated.  Wait for Lumiera to mature.
 +
===Possibly fixable===
 +
*Add YUV4MPEG support in [http://www.blender.org/ Blender] to enable direct output to [http://v2v.cc/~j/ffmpeg2theora/ ffmpeg2theora] ([http://lists.mplayerhq.hu/pipermail/libav-user/2009-March/002639.html email thread]), building upon [[File:Henningsen_7oct2009_blender-svn.tar.bz2|Stephan Henningsen's lead.]] Also build-in yuv4ogg suFigure out a pport as per [http://web.mit.edu/xiphmont/Public/theora/demo8.html xiphmont's lead.]
 +
*Find a plugin/filter that synchronizes audio captured from a secondary source; slight a/v sync drifts accumulate over time, making 2nd-track audio recombination difficult.
 +
*Document export options to accommodate a wider range of video editors; [http://php.scripts.psu.edu/gsc127/blogs/2009/05/firefox-ogg-firefogg-demo.php firefogg] is an option.
-
*Search works but has a few issues (punctuation stripped, 'play inline' plays audio although video is paused, search-by-date not yet enabled for all videos)
+
===Dreamlist===
 +
*Theora encoding ''via'' hardware acceleration (TI C64x+ DSP or an Elphel card?)
 +
*Find a sensible audio compressor to avoid the time-intensive task of adjusting audio to comfortable listening levels.
-
*[http://cinelerra.org/ Cinelerra-CV] is buggy (compilation from source helps). Document rendering options on other video editors (e.g., [http://www.xiph.org/quicktime/ XiphQT]) and help develop [http://lumiera.org Lumiera].  
+
<br>
 +
==Archiving==
 +
===Known issues===
 +
*FTP on the Internet Archive is only available for item download and initial upload; item editing is browser-based.
 +
*Video playback on Internet Archive description pages (inc. <video> fallback) requires the presence of a Flash-based derived file. These derived files are manually deleted post-upload.
 +
*OpenMeetings.org is not crawled by the Wayback Machine.
-
*Fix severe problems with firewire_ohci/firewire_core talking to DV controller chips.  [https://bugzilla.redhat.com/show_bug.cgi?id=449252 Example ticket]
+
===To-do===
 +
*Retool [http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/MetavidWiki/maintenance/download_from_archive_org.php?view=markup download_from_archive_org.php] to work with different source videos and meta tags from the [http://archive.org Internet Archive], using [[OMwiki:Tech/OMwiki-import.php|this wikipage]] as a working draft.
 +
 
 +
===Dreamlist===
 +
*Implement template support on the [http://archive.org Internet Archive] description pages.
 +
*Secure logins + strong item deletion-prevention.
<br>
<br>
-
=='''Scripts needed:'''==
+
==Distribution==
-
*'''Retool [http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/MetavidWiki/maintenance/download_from_archive_org.php?view=markup download_from_archive_org.php] to work with different source videos and meta tags from the [http://archive.org Internet Archive], using [[OMwiki:Tech/OMwiki-import.php|this wikipage]] as a working draft.'''
+
===Known issues===
-
*Draft a script to import MetaVidWiki clips from apps that can produce [[wikipedia:SubRip|.srt]] or [[wikipedia:Continuous_Media_Markup_Language|.cmml]] captions (''e.g.,'' [http://gnome-subtitles.sourceforge.net/ Gnome Subtitles])
+
*Forgo Media RSS ver. 1.5.0 and start on a sensible ver. 1.6. Ver 1.4 is OK.
-
*Rework [http://modcopy.sourceforge.net/ modcopy] to avoid mmap calls on JFFS2 filesystems (''e.g.'', the OLPC-XO). [http://sourceforge.net/apps/trac/modcopy/ticket/9 Ticket]
+
===To-do===
 +
*Implement content delivery network, which will involve site migration.
 +
===Dreamlist===
<br>
<br>
 +
==OMwiki issues==
 +
===Known issues===
 +
*Find a way to add <code>style="width:352px;"</code> to embed code in 352×240 streams and fix aspect ratio for 352×240 thumbnails when in category view.
 +
*Google Chrome has buggy Theora support.
 +
*Edit [[MediaWiki:Common.js]] to download animated gifs only [[OMwiki:Finding_aid|when 'show' is clicked]] to avoid long page loads.
 +
*[[OMwiki:Speakers|Speaker labels]] work for annotations but not transcripts.
-
==New Features==
+
*Search works but has a few issues (punctuation stripped, 'play inline' plays audio although video is paused, search-by-date not yet enabled for all videos)
-
*[http://lists.wikimedia.org/pipermail/metavid-l/2009-August/000055.html Email thread on MetaVid-l]
+
*White-on-black site design works well for focusing attention on video streams (right?), but fares poorly for text-heavy pages.  Implement an automatic "turn down the lights" feature when video is in a 'play' state.
 +
===To-do===
*Add a Vorbis-only option on video streams for low-bandwidth connections.
*Add a Vorbis-only option on video streams for low-bandwidth connections.
 +
*Turn on [http://www.mediawiki.org/wiki/Extension:OpenID OpenID logins.]  Need to figure out how to link non-PEAR libraries to non-recompilable PHP or upgrade web hosting.
 +
*Figure out AJAX auto-complete magic.
 +
*Fix Google search listing.
 +
*Start up project blogging, @openmeetings.org email addresses, and listing of similar projects.
-
*Turn on [http://www.mediawiki.org/wiki/Extension:OpenID OpenID sign-ins.]  Need to figure out how to link non-PEAR libraries to non-re-compilable PHP.
+
===Dreamlist===
 +
*[http://lists.wikimedia.org/pipermail/metavid-l/2009-August/000055.html Ideas on MetaVid-l] (''e.g.'', [http://identi.ca Identi.ca] integration)
 +
*Replace animated GIFs with low-FPS, enlargeable, jump-to-timecode Theora. Alternatively, use [http://pad.ma/ Pad.ma]-style mouse-overs (see [https://wiki.pad.ma/browser/padma.dev/padma/static/javascript/info.js?rev=padma.dev%2C354&order=size&desc=1 <nowiki>$(imagePoster).load(function() {...})</nowiki>]). The thumbnails should cycle through the meeting in large time chunks when non-mouseover'd, and then display clickable frames according the how far the mouse is from the left edge when mouseover'd.  Include ability to pause all animations and step-through lock-step upon mouse clicks.
-
*Replace animated GIFs with low-FPS, enlargeable, jump-to-timecode Theora.  Alternatively, use [http://pad.ma/ Pad.ma]-style mouse-overs (see [https://wiki.pad.ma/browser/padma.dev/padma/static/javascript/info.js?rev=padma.dev%2C354&order=size&desc=1 <nowiki>$(imagePoster).load(function() {...})</nowiki>]).  The thumbnails should cycle through the meeting in large time chunks when non-mouseover'd, and then display clickable frames according the how far the mouse is from the left edge when mouseover'd.
 
-
*Add YUV4MPEG support in [http://www.blender.org/ Blender] to enable direct output to [http://v2v.cc/~j/ffmpeg2theora/ ffmpeg2theora] ([http://lists.mplayerhq.hu/pipermail/libav-user/2009-March/002639.html email thread]), building upon [[File:Henningsen_7oct2009_blender-svn.tar.bz2|Stephan Henningsen's lead.]] Also build-in yuv4ogg support as per [http://web.mit.edu/xiphmont/Public/theora/demo8.html xiphmont's lead.]
+
<br>
-
 
+
==Video post-processing==
-
*Squash bugs (esp., video non-playback) in mwEmbed's [http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/js2/mwEmbed/libSequencer/ libSequencer].  
+
===Known issues===
 +
*No video post-processing is implemented.
 +
===To-do===
 +
*Figure out best options to breakdown tons of video.
 +
*Fixup and advertise availability of [[OMwiki:Nodes|echo.openmeetings.org]] for compute tasks.
 +
*Machine synchronization of known-good transcripts to video (i.e., match individual words to the exact timecode spoken).
 +
===Dreamlist===
*Identify a way to output time ranges generated from multi-speaker audio files, with each time range corresponding to the duration of how long a person spoke for ([http://cmusphinx.sourceforge.net/ CMU Sphinx?]).  Hopefully add speaker identification based on the vocal profile, and maybe even facial recognition and emotional categorization (''e.g.'', laughter).
*Identify a way to output time ranges generated from multi-speaker audio files, with each time range corresponding to the duration of how long a person spoke for ([http://cmusphinx.sourceforge.net/ CMU Sphinx?]).  Hopefully add speaker identification based on the vocal profile, and maybe even facial recognition and emotional categorization (''e.g.'', laughter).
 +
*Have individuals appear as virtual avatars in-video (''i.e.'', Second Life or Avatar-esque), according to user preference.
-
*Figure out a content delivery network and target areas.
 
-
====Dreamlist====
+
<br>
-
*Build open-firmware cameras that record directly to Ogg (2-pass Theora plus Vorbis); also include simplified on-camera editing.  Elphel has completely-open [http://www3.elphel.com/353_turnkey network cameras.]
+
-
*Have those same cameras set to blur-out individuals broadcasting a 'do not record my voice, person, or both' flag.  Via iPhones, RFID, or colored sticker as mentioned in Yochai Benkler's keynote at OVC 2009.
+
-
*Have individuals appear as virtual avatars in-video (''i.e.'', Second Life-esque), according to user preference.
+
-
<br />
+
==External apps/reuse==
 +
===Known issues===
 +
*Wait for mwEmbed's [http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/js2/mwEmbed/libSequencer/ libSequencer] to mature; squash bugs.
 +
===To-do===
 +
*ffmpeg/mplayer seems to handle duplicate Theora frames poorly, breaking A/V sync and thus limiting video reuse.  Submit bug report.
 +
*Draft a script to import MetaVidWiki clips from apps that can produce [[wikipedia:SubRip|.srt]] or [[wikipedia:Continuous_Media_Markup_Language|.cmml]] captions (''e.g.,'' [http://gnome-subtitles.sourceforge.net/ Gnome Subtitles])
 +
*Figure out a easy way to integrate playback with blog platforms, including various commenting systems (''i.e.'', plugins).  [http://www.personal.psu.edu/gsc127/blogs/2009/06/enabling-video-content-on-your.html Known hacks]
 +
===Dreamlist===
 +
 +
 +
<br>
 +
 +
==Minor things==
 +
*USB headsets crash spontaneously (error message <code>usb_set_interface failed</code> in Debian Squeeze).
 +
 +
 +
<br>
==Communicate:==
==Communicate:==
<!-- *Document and test [https://lists.sourceforge.net/lists/openmeetings-discuss project listserv] [https://lists.sourceforge.net/lists/openmeetings-tech (tech listserv)] --><!-- No need for a new list as of 2009 -->
<!-- *Document and test [https://lists.sourceforge.net/lists/openmeetings-discuss project listserv] [https://lists.sourceforge.net/lists/openmeetings-tech (tech listserv)] --><!-- No need for a new list as of 2009 -->
Line 63: Line 120:
<br />
<br />
-
 
==Non-technical development==
==Non-technical development==
*Please see [[OMwiki:Volunteer]]
*Please see [[OMwiki:Volunteer]]

Current revision as of 04:04, 12 March 2010


[edit] Recording devices

[edit] Known issues

[edit] Possibly fixable

  • Rework modcopy to avoid mmap calls on JFFS2 filesystems (e.g., the OLPC-XO). Ticket
  • Investigate audio ground loops and prevention techniques

[edit] Dreamlist

  • Find a portable camera or device that records footage from two audio sources (e.g., XRL mic + ambient mic) time-synchronized to a video track.
  • Migrate to open-firmware cameras that record directly to Ogg; also include simplified on-camera editing. Elphel has completely-open network cameras supported by the Apertus community.
  • Auto-balancing, wireless, golfball-sized mics that are mutable by the camera. Best for when push-to-talk mics are not available.
  • Have cameras set to blur-out individuals broadcasting a 'do not record my voice, person, or both' flag via iPhones, RFID, or colored stickers as mentioned in Yochai Benkler's keynote at OVC 2009.


[edit] Editing

[edit] Known issues

  • Cinelerra-CV is buggy and somewhat dated. Wait for Lumiera to mature.

[edit] Possibly fixable

[edit] Dreamlist

  • Theora encoding via hardware acceleration (TI C64x+ DSP or an Elphel card?)
  • Find a sensible audio compressor to avoid the time-intensive task of adjusting audio to comfortable listening levels.



[edit] Archiving

[edit] Known issues

  • FTP on the Internet Archive is only available for item download and initial upload; item editing is browser-based.
  • Video playback on Internet Archive description pages (inc. <video> fallback) requires the presence of a Flash-based derived file. These derived files are manually deleted post-upload.
  • OpenMeetings.org is not crawled by the Wayback Machine.

[edit] To-do

[edit] Dreamlist

  • Implement template support on the Internet Archive description pages.
  • Secure logins + strong item deletion-prevention.



[edit] Distribution

[edit] Known issues

  • Forgo Media RSS ver. 1.5.0 and start on a sensible ver. 1.6. Ver 1.4 is OK.

[edit] To-do

  • Implement content delivery network, which will involve site migration.

[edit] Dreamlist


[edit] OMwiki issues

[edit] Known issues

  • Find a way to add style="width:352px;" to embed code in 352×240 streams and fix aspect ratio for 352×240 thumbnails when in category view.
  • Google Chrome has buggy Theora support.
  • Edit MediaWiki:Common.js to download animated gifs only when 'show' is clicked to avoid long page loads.
  • Speaker labels work for annotations but not transcripts.
  • Search works but has a few issues (punctuation stripped, 'play inline' plays audio although video is paused, search-by-date not yet enabled for all videos)
  • White-on-black site design works well for focusing attention on video streams (right?), but fares poorly for text-heavy pages. Implement an automatic "turn down the lights" feature when video is in a 'play' state.

[edit] To-do

  • Add a Vorbis-only option on video streams for low-bandwidth connections.
  • Turn on OpenID logins. Need to figure out how to link non-PEAR libraries to non-recompilable PHP or upgrade web hosting.
  • Figure out AJAX auto-complete magic.
  • Fix Google search listing.
  • Start up project blogging, @openmeetings.org email addresses, and listing of similar projects.

[edit] Dreamlist

  • Ideas on MetaVid-l (e.g., Identi.ca integration)
  • Replace animated GIFs with low-FPS, enlargeable, jump-to-timecode Theora. Alternatively, use Pad.ma-style mouse-overs (see $(imagePoster).load(function() {...})). The thumbnails should cycle through the meeting in large time chunks when non-mouseover'd, and then display clickable frames according the how far the mouse is from the left edge when mouseover'd. Include ability to pause all animations and step-through lock-step upon mouse clicks.



[edit] Video post-processing

[edit] Known issues

  • No video post-processing is implemented.

[edit] To-do

  • Figure out best options to breakdown tons of video.
  • Fixup and advertise availability of echo.openmeetings.org for compute tasks.
  • Machine synchronization of known-good transcripts to video (i.e., match individual words to the exact timecode spoken).

[edit] Dreamlist

  • Identify a way to output time ranges generated from multi-speaker audio files, with each time range corresponding to the duration of how long a person spoke for (CMU Sphinx?). Hopefully add speaker identification based on the vocal profile, and maybe even facial recognition and emotional categorization (e.g., laughter).
  • Have individuals appear as virtual avatars in-video (i.e., Second Life or Avatar-esque), according to user preference.



[edit] External apps/reuse

[edit] Known issues

[edit] To-do

  • ffmpeg/mplayer seems to handle duplicate Theora frames poorly, breaking A/V sync and thus limiting video reuse. Submit bug report.
  • Draft a script to import MetaVidWiki clips from apps that can produce .srt or .cmml captions (e.g., Gnome Subtitles)
  • Figure out a easy way to integrate playback with blog platforms, including various commenting systems (i.e., plugins). Known hacks

[edit] Dreamlist


[edit] Minor things

  • USB headsets crash spontaneously (error message usb_set_interface failed in Debian Squeeze).



[edit] Communicate:



[edit] Non-technical development

All videos and text are published under the CC-BY 3.0 U. S. or CC-BY-SA 3.0. copyright licenses.  Details.