Patents, Video, and an Open Internet

For a number of reasons, I’m fascinated by the fight over the <video> tag in HTML5 as related by Ryan Paul of Ars Technica – and not just because I like the idea of not having to install a plugin to watch video online.

On the technical side, it’s mind-boggling to think about the possible consequences of some of these decisions. You have Google suggesting that the wrong codec would demand more bandwidth to run YouTube than is available on the entire Internet. That’s a big number. I am sincerely glad I’m not the engineer who has to manage changes at that scale.

More optimistically, you have the prospect of having native support for video in every browser, without paying or contracting with Adobe for the privilege. That’s exciting.

The friendly rivalry between Theora and H.264 is also neat to watch. I think it’s great that the Theora folks are responding to Apple and Google’s quality and performance concerns. It sounds like addressing their objections has made for a better standard.

Now for the ugliness. The patent portion of this fight is completely ridiculous. As I understand it, you have Apple and Google advocating for the compulsory use of a patent-encumbered and decidedly proprietary codec. In other words, they are putting every producer and broadcaster (which includes individuals) of video on the web in a position where they must pay rent to the MPEG LA consortium. The current licensing terms do not require this, but the license is due to be renegotiated in 2010 and we can guess how that will turn out, based on Jan Ozer’s conversation with Allen Harkness, director of licensing for MPEG LA:

“When I spoke with Harkness, he stated that the patent group hadn’t yet decided the license provisions for internet broadcast, or even if there would be a license, though he conceded that it would make little sense for the patent group to forego this revenue. The only thing certain is that the royalty provisions must be announced by January 2010 for royalties that would be payable the following year.” (emphasis mine)

And who among us can watch as these decisions get made by the MPEG LA? Presumably, those who can log in to the MPEG LA Meetings page. I’m not interested in that kind of uncertainty in a basic building block of the average user’s Internet experience. It provides the MPEG LA a de facto monopoly on online video. Worse, this could raise the bar unnecessarily high for an HTML5-compliant open source browser. If our goal is a free and open web, I don’t know why this is even on the table. I sincerely hope I’m misunderstanding the issue, because it seems the consequences of a H.264 <video> tag are dire.

In light of this, Ogg Theora seems like a good alternative as it doesn’t seem to be encumbered by patents. Google and Apple, though, are not so sure. They believe Theora hasn’t been cooking long enough to draw the attention of any submarine patent holders. Theora advocates turn this argument around and suggest that H.264 could also have undiscovered patent encumbrances. I appreciate the logic of what they’re saying, but the fact that H.264 is already so wildly popular for so long seems to guarantee that any patent trolls would have surfaced by now.

In any case, this is a great illustration of just how profoundly broken the patent system is, with respect to software. Instead of encouraging innovation, it is stifling it.

10 responses

  1. I think you will find people *now* that will write up patents on extensions to theora. All they have to do is to figure out how theora might change and patent the “deltas”. Worse still, they can be very general in their patent descriptions.

    So while theora devs gave away everything, the leeches that come later want to encumber everyone, and they aren’t held back by prior art (existing theora) because prior art is, by definition, given away for free. Thus, the leeches who do little work can block everyone and are entitled to unconstitutional monopolies while the smarter and nicer folks don’t have levers to fight back (except to invent another *gift* again that can again be used against them and everyone else). If patents worked like copyright (automatic), it would quickly become clear that the patent system is unworkable except maybe for high manufacturing cost goods and such (because of the limited participation and access by inventors to those fields).

    Google and Apple probably will get a sweet deal (very nice cut) from the MPEG people if they can grow that market. Expect these two to be among the batch of leeches that helps fund patents to encumber theora.

  2. the fact that H.264 is already so wildly popular for so long seems to guarantee that any patent trolls would have surfaced by now

    Actually, that’s not true, and we had already a case like that: GIF.

    Even if it’s wildly popular, one sensible strategy for a patent troll would be to wait until it is a real de facto standard (implemented natively by many browsers and having vasts amount of content encoded with it) and THEN start demanding. Right now, it is somewhat replaceable. Down the line, it will be harder.

    That’s one of the main issues in this debate.

    @emailtoid-guy, although particular extensions could indeed step into patents, that risk is no bigger than what’s already built to step on them. And the Theora guys have experience avoiding that. And I don’t understand how, by definition, no-cost software does not count as prior art. In the case of Free software (and all Theora implementations by now are Free), I believe it is easier for them to count as prior art (as there are tons more information about them posted earlier).

    nachokb

  3. >> And I don’t understand how, by definition, no-cost software does not count as prior art.

    What I said (or intended to say) was that FOSS (for the most part) counts at most as prior art. So FOSS gives away while others leverage it and then put a stop for everyone else. If patents were automatic like copyrights (and I know that patents are intentionally as vague and general as is the skill of the patent author), then all the FOSS being made would be giving all of these proprietary companies lots of headaches and patent risks. The result would likely be to isolate the pure leeches (trolls) as the only patent winners. Instead you have large proprietary companies likely looking at patents as a way to beat down smaller competitors (or let the trolls/proxies beat them down for them).

    >> although particular extensions could indeed step into patents, that risk is no bigger than what’s already built to step on them.

    What I have noticed is that most patent claims are light: they extend a quasi involved patent claim in almost trivial ways (certainly by adding a lot fewer requirements/properties).

    If “my” main patent claim is designed to cover many aspects of (eg) theora plus some simple extension that might arise naturally in the future once certain types of services or what not takes hold (and for which today little if any prior art can be found), then that quasi involved claim not only might stand up, but it becomes easy to just add on features on a per claim basis. These other (extension) claims are even more likely to stand up in court.

    Since patents can have claims fail while others succeed, I presume you can even have your first claim be nothing but a generalized description of (eg) theora (doomed to fail in court) while the additional patent claims add the little extensions (and likely be upheld in court). An “extension” can be anything that you may very well want to do with theora some day but actually haven’t yet done (so no prior art would exist currently).

    Patents are very general/broad. They truly stifle by laying claim to too much (much more than copyrights). Here: http://www.freepatentsonline.com/y2007/0035109.html?query=car&stemming=on
    http://www.freepatentsonline.com/7565404.html
    http://inventors.about.com/od/patentsbasics/a/PatentClaims.htm (see the example about the tent)

    Note, how general these are and how most claims add rather trivial extensions.

    If more ordinary people and developers knew what a hoax patents were (especially software patents) and the threat these pose today to FOSS, they would be rather angry and might even actively rebel against the patent system.

    And before you talk about obviousness look at some patents to see that many of them are so general that sometimes grade school kids could come up with them (in art class even).

    BTW, I am not trying to imply that theora is as bad as any other. In fact, it would be shameful for someone to do what I described above and then show their face in public. But given how straightforward is the process (and I don’t say this from experience, but you can judge for yourself on the evidence you come across), I expect for enough money some people will seek those patents.

  4. nachokb, you make a great point about GIF. Certainly, it’s not unheard of. But I’m under the impression that the longer something is in the wild, the lower the risk. I don’t think there’s a way to guarantee, forever, that something is completely unencumbered — but longevity seems to help.

    Your larger point about the dangers of pinning all our hopes on a single codec are well-founded, of course. The trouble is that there doesn’t seem to be a viable alternative right now.

  5. My point in the second part of the last comment may not have been clear. Let me try again.

    >> although particular extensions could indeed step into patents, that risk is no bigger than what’s already built to step on them.

    It is more difficult — not equally difficult (in my layman’s opinion) — to capture in a patent the main structure (ie, theora) from nil (ie, without knowing about theora) than it is to capture the main structure plus a new improvement when you already know to start off from that main structure (and not from nil).

    Yes, theora already was dealing with lots of patent mines, but it’s different because the devs could afford to change any requirement at all of any of those patents to avoid the patents. Once they felt safe, they could then standardize/stabilize the format.

    Meanwhile, a patent extension based on theora cannot be bypassed so easily by the theora devs because, for the sake of backwards compat, you cannot afford to change anything that is convenient to change as they did when they were initially developing theora (since the most convenient changes likely are arbitrary details but which could be changed only while breaking backwards compat).

    Additionally, before theora was a target, they had more room to work with. Once a target, even if they are willing to keep breaking the older formats, it will become more difficult to avoid attacks (that is not to say that most these extension patents will be announced publicly anytime soon).

    It’s not hopeless, and I do agree the theora people are in much better shape than those that are already deep in patent trouble, but the point is that the patent system is wholly unfair and can create real problems the way I mentioned.

    And, no, you do not know what patents are out there, so certainly theora might have some claims currently (but this was not the point I was trying to make). Software patents stifle in many ways even when they involve insight originally, but there are some *really* horrible problems with the system, eg, as mentioned above: it is unfair to socially generous and hard-working/creative FOSS devs while enriching greedy (and at times almost) brainless leeches.

    Copyright, which already apply to all software, is plenty general in scope. They may even be too general in scope (“derivative work”).

    Patents.. patents are a bad joke gone mad.

  6. @emailtoid-guy (sorry, what’s your nick?) you’ll have to give me a little time to read that :D

    @gunnar, I still disagree, particularly,

    the longer something is in the wild, the lower the risk

    That would be the norm in a sane world with sane laws. But that’s just not how patents work (software patents at the very least).

    Everyone involved with software patents is trying to game the system: trolls to earn money for almost nothing, big companies to defend themselves, monopolists to tilt the playing field.

    It’s not enough for something to be a long time in the wild, it also has to be “attractive”. If certain format would be a potential victim to a patent attack, but nobody uses it, the only incentive for the patent owner to sue is to not lose the rights to the patent (and they can always claim ignorance). So the more opportunity to extract juice from the patent, the more likely implementors of the format are to be sued.

    The problem lies in the fact that, when such a format is a candidate to be included in some universal standard, the potential juice to extract is greater if you wait until it is accepted…

    nachokb

  7. >> And I don’t understand how, by definition, no-cost software does not count as prior art.

    Let me give an example of how having prior art (but no patents) is weak in a world of patents.

    Person A patents a basic car (wheels, surface to hold people or things, windows, roof, doors).

    Person B patents a car that also has rear view mirrors (or just patents the mirrors for use as such on cars).

    Person C patents a covertable/foldable roof.

    Person D patents the use of air conditioning.

    Person Z patents some other great thing you might want in a car (eg, radio).

    Let’s now look at a typical person in the market to buy a car. Unless that person wants a car as bland as warm tap water, they will have to deal with a bunch of patents. The person will pay A and a bunch other patent holders (at least they will indirectly).

    What about person A? Well, person A can prevent all others from building their cars if person A so wishes (assumption here is that all other patents would violate or require the basic car invention), or person A might allow others to build their inventions for a royalty (but then A might have to net pay out if A wants anything but a bland car).

    Now, what if person A did not have a patent but instead had nothing but a great idea and prior art?

    Well, person A could prevent someone else from patenting that basic car, but A could not stop B or C or …Z from patenting their more elaborate car (or uses of, adjuncts to, … a car). So if A ever wants a cool car, they will have to deal with the terms of these other patent holders and without any leverage (meaning that these other people could simply forbid A from using or creating those inventions.. even though all of those inventions extend the basic car A invented.

    To get back to theora. A patent on using theora streams with movies that are displayed on the sides of a 3D spinning cube might not have prior art (actually it probably does, today). This means someone could (have) patented that (at least before prior art existed or could be proven to have existed 10 years from now when they go to court). Another patent might cover theora holding content from multiple urls where one of the urls provides content at an improved quality. Well, even if theora supports that today (?) it might not have prior art, today, so a patent could be taken out. And if that is not supported today in the standard as a possibility, it might be a natural thing to add to theora tomorrow, but a patent writer targeting theora can beat them to it (before there is prior art) and write the patent in a very general way so that you can’t bypass it if (a) you use core theora as it exists today and (b) you want that new functionality/extension/use for theora tomorrow.

    Remember, the rear-view mirror, air conditioner, and radio were all possibilities on the basic car invention from day one, but until someone actually puts that together, it is open to being patented.

    >> @emailtoid-guy (sorry, what’s your nick?) you’ll have to give me a little time to read that

    I spent some time looking at this issue over the past year, as I had concerns over dotnet/mono. [It's a "standard" some say.. the core basic bland invention is "safe" so we can do anything with it some say.. we can code around it later in the libraries if patents show up some say.. the MCP is a good promise that actually safeguards more than a doorknob some say.. OK, now I'm really getting off topic.. sorry.]

    What I have seen with software patents is disgusting.. offensive to the reasonable mind.

    – Jose_X

    PS: Future court rulings may remove the greater obscenities from patent law (eg, protect theora or even FOSS), but the more you play with (a hostile patent writer’s) fire, the greater the chance no court will ever save you or save you from major headaches or costs.

  8. [This comment is an edited version of the last one.. I submitted the other one unintentionally. A few of the middle paragraphs are a little different and perhaps clearer.]

    >> And I don’t understand how, by definition, no-cost software does not count as prior art.

    Let me give an example of how having prior art (but no patents) is weak in a world of patents.

    Person A patents a basic car (wheels, surface to hold people or things, windows, roof, doors).

    Person B patents a car that also has rear view mirrors (or just patents the mirrors for use as such on cars).

    Person C patents a covertable/foldable roof.

    Person D patents the use of air conditioning.

    Person Z patents some other great thing you might want in a car (eg, radio).

    Let’s now look at a typical person in the market to buy a car. Unless that person wants a car as bland as warm tap water, they will have to deal with a bunch of patents. The person will pay A and a bunch other patent holders (at least they will indirectly).

    What about person A? Well, person A can prevent all others from building their cars if person A so wishes (assumption here is that all other patents would violate or require the basic car invention), or person A might allow others to build their inventions for a royalty (but then A might have to net pay out if A wants anything but a bland car).

    Now, what if person A did not have a patent but instead had a great idea that would establish prior art when built? [theora scenario]

    First note that it is much easier and quicker to get a patent once you have the idea and basic outline than it is to actually build it (building well is the harder job, especially because so many details are involved in building that are not put into patent claims in practice.. and to debug, etc).

    Well, person A could prevent someone else from patenting that basic car, but A could not stop B or C or …Z from patenting their more elaborate car (or uses of, adjuncts to, … a car). So if A ever wants a cool car, they will have to deal with the terms of these other patent holders and without any leverage ..meaning that these other people could simply forbid A from using or creating those inventions.. even though all of those inventions extend the basic car A invented!

    To get back to theora. A patent on using theora streams with movies that are displayed on the sides of a 3D spinning cube might not have prior art (actually it probably does, today). This means someone could (have) patented that (at least before prior art existed or could be proven to have existed 10 years from now when they go to court). Another patent might cover theora holding content from multiple urls where one of the urls provides content at an improved quality than the other. Well, even if theora supports that today (?), this functionality/use might not have prior art, today, so a patent could be taken out. And if that is not supported today directly in the spec (let’s assume), it might be a natural thing to add to theora or standardize precisely tomorrow, but a patent writer targeting theora can beat them to it (before there is prior art) and write the patent in a very general way so that you can’t bypass it if (a) you use core theora as it exists today and (b) you want that new functionality/extension/use for theora tomorrow. You could not avoid this general feature no matter how you implemented it because you would still be implementing that general feature as described generally in such a patent. [Look at patent claims to see how general they are. I'm not joking. They patent the mere existence of something when that something is missing almost all details.]

    Remember, the rear-view mirror, air conditioner, and radio were all possibilities on the basic car invention from day one, but until someone actually puts that together, it is open to being patented.

    >> @emailtoid-guy (sorry, what’s your nick?) you’ll have to give me a little time to read that

    I spent some time looking at this issue over the past year, as I had concerns over dotnet/mono. [It's a "standard" some say.. the core basic bland invention is "safe" so we can do anything with it some say.. we can code around it later in the libraries if patents show up some say.. the MCP is a good promise that actually safeguards more than a doorknob some say.. OK, now I'm really getting off topic.. sorry.]

    What I have seen with software patents is disgusting.. offensive to the reasonable mind.

    – Jose_X

    PS: Future court rulings may remove the greater obscenities from patent law (eg, protect theora or even FOSS), but the more you play with (a hostile patent writer’s) fire, the greater the chance no court will ever save you or save you from major headaches or costs.

Comments are closed.