Nowadays video content becomes more and more popular on the Internet. Social networks as well as corporate web sites require hosting of video content and playing it online.

The major problem for video content on the Internet is that current internet browsers do not include any video decoders. So if you want to play video online you will have to install some stand-alone platform. I would highlight two major platforms allowing playing video movies:

1. Adobe Flash.
2. Microsoft Silverlight.

Let us consider these two platforms in terms of video support in detail.

Microsoft Silverlight

We all know that Silverlight is officially supported on Windows and Mac platforms. Also there is Moonlight project which is Silverlight implementation for Linux platforms. Starting from the very first version Silverlight supports decoding of WMV format. Moreover, Microsoft has opened WMV decoder source code and granted its usage in Moonlight. So decoding of WMV format is supported on the major computer platforms.

Adobe Flash

During the last years Flash became standard de-facto for playing video on the Internet. The most of users know the video format supported by Flash as FLV but it is not exactly true. FLV is the video and audio container used to deliver video and audio content from server side to the client. The format of this container was opened by Adobe on May, 1, 2008. The container itself can contain the following video formats:

1. Sorenson spark, minimum required flash player version 6.
2. On2 VP6, minimum required flash player version 8.
3. H.264, minimum required flash player version 9.

Now we know what video formats are supported by Flash and Silverlight. Let us examine licensing policies for them.

We get used that image formats usually have open format and reference encoder libraries. You can use these encoders for free. Video formats unlike images get the problem here. They all are patented and have an interesting trick. Decoding is free for end users and does not require any royalties though encoding is not royalty-free. So even if you get Mpeg4 format reference (H.264) and implement encoder by yourself, you will have to pay royalties for its usage. So then…

Video Formats

WMV format

It is Microsoft format for storing and streaming video widespread on Microsoft platforms. There is free for use Windows Media Encoder which allows encoding to WMV. The only limitation is that it works on Windows only.

Sorenson Codec

This video Format is extension for H.264 designed by Sorenson Media, the specification is not opened. The encoder is the part of Sorenson Spark SDK. It seems that this codec was licensed by YouTube and Google. Sorenson do not publish the pricing on their site but anyway it is not a free encoder.

On VP6

Like Sorenson Spark On2 VP6 is the extension for H.264 designed by On2 Technologies, the specification for this format is not opened too. This company sells encoders for this format as part of On2 VP6 Codec SDK and Flix SDK for Direct Show. On2 do not publish pricing on their site as well.

H.264

It is known as Mpeg4 format which is standard de-facto in the world of digital video. In contrast to the previous formats it has opened reference. Nevertheless it is heavily patented and there is a special company MPEG LA that manages licensing royalties for this format. We examined licensing policies and found that there are two major options:

1. For decoder/encoder suppliers who integrate them in end-user solutions.
2. For service providers like IP TV and so on.

In both these cases royalties are required either for each copy of product or for each movie transcoded and streamed server-side. There are a lot of codec suppliers for Mpeg4 format but it seems that they provide implementation only. The licensing stuff is a headache of the company which purchases the codec and wants to use it in its solution.

Video Uploader

Summarizing, we all understand that there are a lot of video formats worldwide in addition to listed ones. Nevertheless, if we want to play movies via the Internet we have to transcode them to formats that can be played back in internet browsers (using Silverlight of Flash). And the core feature of our Video Uploader is transcoding of video movies to a single format relieving server platforms of encoding operations. This way all operations with movies are shifted from server side to the clients simplifying server logic and decreasing server hardware requirements. We have chosen WMV format. Why? Since you do not need to pay anything if you use it on Windows platforms. Today when we released beta version of Video Uploader ActiveX and started getting the feedback from you, our customers, we are searching the way to implement Java version of the product and decide on the question with video-related licensing for our customers.

 

Recently we have decided to widen our experience (and maybe products line) and to investigate new Flash/Flex platform. The task was to create file-upload solution with some client-side image processing. This post contains a summary of our experiments. However, I'm a full newbie in Flash/Flex, so feel free to add your notes and comments.

The beginning was great - I have just opened FlexBuilder and found many cool things. Stable and fast visual forms designer, easy-to-learn syntax, all major OOP concepts and even compile-time type checks. Oh, and of course - shaders! Really amazing stuff. However, after few days my impressions were not so bright. Why? Here the whole list of reasons.

Security restrictions

It is really a problem number one. As I was interested in upload functionality, I started with URLLoader class. It cannot post anything to server if this action is not initiated by user directly. Very reasonable. So, each time when you call URLLoader.load() you should have something like button click handler higher in stack. It prevents malicious apps from stealing user data. But...

There is another thing about Flash/Flex which you should know. It is event-driven platform with many asynchronous calls. E.g. if you want to load file - you should call FileReference.load() and wait for "complete" event fired by this object. If you want to load SWF movie or JPEG image - you should create Loader object, ask it to start processing and just wait for the event. Very simple! If you are not going to upload an image you have, to go to fullscreen or do anything else from the list of restricted actions. But if you are - you are in a big trouble. By the reason of the fact that "complete" event fired from another thread - there is no UIA (user-initiated-action) handler in stack and, hence, you have no permission.

Adobe docs says - "you should show message and ask user again". You have to ask, even if a user just clicked "Do this" button just because you have asynchronous call in the middle of your operation. Really annoying. Actually, I don't understand why a bad guy cannot show message and ask user some innocent question? User will give him a permit (why not, if question is innocent?) - and he will do his dirty business...

And there is also another bug with URLLoader - it doesn't fire progress event while uploading data from binary buffer. It is a known issue and I even voted for it. I would recommend you to do the same if you experience a similar problem.

So, URLLoader was evidently not the best choice. Next thing I tried to use was Socket. Yeap, it is not so easy to use, however, from the other hand it gives much more flexibility. Nevertheless, a few hours later it was evident that Socket is not the option at all because even if you are going to make uploads to the same domain where you host your SWF file you should have special policy server installed on the server. You cannot obtain permission policy from your HTTP server, no, only via special policy server...

Threads & Asynchronous calls

As I mentioned before - Flash/Flex is an event-driven asynchronous platform. But it doesn't support multithreading. Sounds strange, yeap? As far as I understand from blogs and forums - all ActionScript code is executed in a single thread. And asynchronous calls are just enqueued in some internal dispatcher. What does that mean? That it is simple to develop, it is hard to make a hardly-debuggable synchronization error. And this means, that you cannot run anything in background. There are workarounds which emulate "background processing", but they were not applicable in my case. Oh, I forgot to mention that shaders engine works in another thread, so if your task can be expressed in shaders programming language - you will be able to do it asynchronously and very-very fast. ;) Unfortunately, it isn't my case as well.

Summary

What do we have at the end? A very strange feeling. Get me right, I like Flash/Flex - it provides a bunch of great features, it allows to develop really fast, it works in a stable way... But sometimes it stalemates you, and you cannot do anything. Users will have nothing to do than bear user interface freezing or answer useless questions shown because of security restrictions in Flash.

It would be great to have the following things in the future:

• Signed SWFs or any other mechanism to use instead of UIA concept. Or some mechanism which will allow passing UIA-permission via event chains.
• Multithreading or at least something like yield() method.
• Ability to work via sockets. At least with native domain. Without any "special server on XXX port", because it is not usable in real world with hosting providers, firewalls, and proxies.

At the Max 2007 developer conference Adobe product manager Geoff Baum gave a demo of Photoshop Express, the Flash-based image editor that runs inside a Web browser: http://www.news.com/Adobe-flashes-more-looks-at-online-Photoshop-Express/8301-10784_3-9790168-7.html?part=dht . This press-release says that PhotoshopExpress will targeted to low-end market to compete with free desktop photo editors. This product has the following functionality:

• red eye removal
• thumbnail generation
• varoius photo effects like sepia
• interactive undo/redo
• color correction
• and so on

This press-release http://www.news.com/Adobe-to-take-Photoshop-online/2100-7345_3-6163015.html says that the product will be free, Adobe is going to monetize it using online advertising.