Changing Concepts of Time
It's the greatest and most spectacular battle in the Lord Of The Rings Trilogy. But during its production filmmakers faced one surprising challenge how to keep the computer-generated soldiers from fleeing the battlefield. Director Peter Jackson had laid down his requirements for the Battle of Pelennor Fields… Jackson wanted the computer-generated antagonists to
have absolute authenticity on the big screen and to be indistinguishable from the real actors. "I want battles like nothing anyone has ever seen on screen I want every soldier fighting for himself… you have to come up with something."… Special effects designer Richard Taylor says Jackson‟s order led to the writing of a "massive" principal code for the battle in order to give the more than 200,000 digitized soldiers and some 6,000 horses distinctiveness and individuality… "It was the fact that you could get a computer to think for itself, and that you could also get 200,000 agents within the computer to think for themselves. So each of these computerized soldiers is assessing the environment around them, drawing on a repertoire of military moves that have been taught them through motion-capture determining how they will combat the enemy, step over the terrain, deal with obstacles in front of them through their own intelligence and there's 200,000 of them doing that. Basically, all the necessary information for decision-making was fed into this network of computers without determining for them whether they would win or lose. But this aim to ensure that they acted spontaneously almost sabotaged the battleground sequences. For the first two years, the biggest problem we had was soldiers fleeing the field of battle," Taylor laughs. "We could not make their computers stupid enough to not run away." So some extra computer tinkering was required to ensure that the trilogy's climactic battle worked the way Jackson wanted.
Globe and Mail – March 2004
(Pat Crawley Lughnasia 2005)
Production Network Experiment
The 2k Production Pipeline ------------------------------------------- page 3
Servers and Network Infrastructure --------------------------------- page 8 9/11 and Network based Production --------------------------------page 18 E-cinema on Location ------------------------------------------------ page 37 Motion-control and Performance-capture ------------------------- page 41 Virtual Art-direction ------------------------------------------------- page 53 Cost-control and Non-linear Production -------------------------- page 61 Institute for Creative Technologies -------------------------------- page 71 Machinima ------------------------------------------------------------- page 76 Computer-aided-design or CAD ------------------------------------ page 87 Flattening the Production Unit -------------------------------------- page 93 Rendering Farms and Grid-computing ---------------------------- page 100 DreamWorks and HP ------------------------------------------------ page 109 IBM, Disney and Others --------------------------------------------- page 111 Super-computers, Animation and Special-effects ---------------- page 114 Arrested Development and Runaway Production ---------------- page 121
Production Network Experiment
If we were to visit Sam Goldwyn's Metro-Goldwyn-Mayer lot in the 1930s and watch a movie being made, there would be very, very little difference in the rhythms, in the work, in the activity done on that lot than there is in doing a modern television movie. In spite of the fact that the machines are different, the rhythms and patterns and the time it takes, and the number of people, have basically not changed in 60 years. 60 years in which technology has reduced the cost, and increased the diversity of every other consumer product, from cars to recorded music to computers, making those products objects of easy access and great variety. When we look at the way satellites, microwaves, and fibre optics have conquered space, and conquered time and then we look at how television production is done. We cannot conquer weight and we cannot conquer light. Why is it that so much of television costs come in simply setting up lights? Why is it that so much of journalism costs consist of schlepping heavy stuff onto airplanes and off airplanes? Clearly the main problem we have with television production is its cost. The most important part of our industrial strategy must be to reduce the cost of production. And I look to technology as the great unexplored frontier. I am convinced that there is a technological way, of challenging volume, of challenging weight, and challenging the idea that television production has to take such a long time and take so many people, and be so heavy! One way we will get the cost of production down is to look towards the technical solution and that technical solution is not beyond our capacity.
Trina McQueen – 1992
The 2k Production Pipeline
Digital technology has actually been impacting the film and television
production, transmission and display systems in serious ways since Silicon
Graphics deployed powerful workstations applied to 2 and 3d computer-
graphics and high resolution electronic image processing into the operational components of these strategic industries on a world wide basis These technologies nudged the operational side of these strategic industries into what would turn out to be an inevitable transition from analog and non-interactive towards digital and interactive formats following a path already well traveled by the military in the US during the 60s, 70s and 80s. When via the space program, various other surveillance, intelligence-gathering, mapping and communications activities of the Defense Department, National Reconnaissance Office, National Security Agency and Defense Advanced Research Projects Agency generated the key concepts and chipsets necessary to the digital technological revolution in general.
The peace dividend shifted these disruptive technologies into the civilian side of the IT and consumer-electronics sectors sending them spinning off in a search of fresh markets and cash flows within the entertainment industry in general and the media sector in particular. This placed the components of the media sector able to best exploit the incoming technological base at the centre of a revolution in electronic information-processing, transmission and display. A transformation fueled by the super profits generated by from the commercial aspects of the soft-power strategy as accelerated by the dot.com investment bubble, globalization and stimulated after 9/11 by the Bush Administration‟s expanded propaganda and disinformation requirements resulting from the so called “War on Terror.”
The powerful chipsets when combined with the emerging 3d
applications enabled the computer-graphics industry to transcend its scientific and military origins; overcome barriers of feasibility, cost-effectiveness and productivity; to become pervasive in all aspects of narrative and interactive media design, production and distribution while simultaneously enabling the field of computer-aided-design in general.
The cash flows generated at the box office by feature films including ever more computer-graphics plus the sales of DVDs, video-games, digital, HD and now IP based television signals expanded the cash flows available to the already digital sides of the sector to the point where in the late 90s they surpassed those of the still analogue elements of the marketplace. While at the same time the interactive products such as video games and IP based television signals were also setting the tone within the film, television, animation, gaming and multi-media marketplaces on a world wide basis
from an aesthetic point-of-view. As evidenced in the growing marketplace for, and apparent cultural importance of videogame hard and software, as well as, the various peer-to-peer file transfer systems which disrupted the music industry to the point of forcing it to adopt an interactive and transactional business plan connected with the arrival of the iPod and Podcasting.
Provocatively, in the whole area of software for 2 and 3d computer-graphics, the core technologies necessary to both videogames and computer-generated animation and specific effects industries. It was Canadian companies leading the pack on a world wide basis. For example Softimage and Discreet Logic both of Montreal, and Alias Wavefront of Toronto, by
now all owned by important American hard or software interests but still head-officed in Canada engineered the software-environments necessary to the computer-graphic side of a high-resolution electronic image-processing-revolution on a world wide basis.
Alias Wavefront was formed in Toronto after a 1982 visit to Industrial Light & Magic (ILM) inspired Ottawa‟s Stephen Bingham to create an easy-
to-use 3d software package for use within the productive components of the film and broadcast industries. In 1984 Alias suggested to SGI that their mini-computer would form an excellent platform for computer-aided-design based on Alias‟s emerging 3d software package. In 1986 Softimage was
founded in Montreal by an ex-NFB filmmaker based on the idea of creating a 3d animation system for application to the media production for use by artists and filmmakers. Like Bingham, Daniel Langlois was dissatisfied with the existing 2 and 3d graphics packages designed primarily with military or scientific applications in mind.
Since off-the-shelf software for the production of films, television programmes and videogames were not available the founders of these Canadian companies, along with Wavefront Technologies in California, since merged with Alias, adapted their business plans to develop and market their own 2 and 3d computer-graphics packages. Designed primarily for use by artists working within animation, video-gaming, film production, broadcast and computer-aided-design or the CAD sector in general.
In 1987 the Belgian government provided the capital to enable Alias to focus on the computer-aided-design side of the business and they soon developed an impressive customer list including Timex, Reebok, BMW, GM,
Honda, Volvo, Apple, GE, Motorola and Sony. As well a growing client list in the entertainment industry which grew to include Industrial Light & Magic, Broadway Video, Moving Picture Company and ultimately , Dreamworks and Pixar. In 1989 Steve Williams (ex-Alias) went to ILM to help another Canadian James Cameron creating the pseudopod in The Abyss. Alias was chosen because of its patch instead of polygon based modeling system. The software ran on SGI workstations and the Abyss garnered an Oscar. In 1990 Alias went public and introduced a third generation of software branded Studio for industrial design and PowerAnimator within the entertainment industry in general.
In 1991 Softimage released it‟s second-generation 3d-graphics
package featuring an “actor module” based on inverse kinematics and enveloping constraints enabling animators to combine conventional techniques such as editing and keyframing with advanced computer-graphic tools of various kinds. The package heralded a revolution in creating realistic character motion and the assembly of the very first digital actors. In 1991 Wavefront launched Composer that quickly became the standard for professional 2 and 3d compositing and special effects for the film industry while heading the optical-printers, scanners and film-recorders off in digital directions, as well as, setting the stage for the whole idea of digital intermediates (DI) in general.
Concurrently, Alias announced an alliance with SGI and purchased Spacemaker Technology and launched UpFront a low-cost 3d Mac and Windows based package for architects. Alias also broadened it‟s product range with the acquisition of Sonata, a high-end 3d architectural design and presentation system from T2 Solutions in the UK. In spring of 1992 Alias also introduced an upgraded version of PowerAnimator including Kinemation and SmartSkin a “character animation” system for natural motion and muscle behavior. Dynamation was a 3d tool for interactively creating and modifying realistic, natural images of dynamic events. The system allows the seamless blending of behavioral data and user-specified information describing shape, color and motion. The tool set needed to first create digital performers and then to release them into digital so-called “virtual realities” were thus at hand.
In 1992 Softimage went public and acquired EDDIE software and Painterly Effects, providing a complete effects generation toolkit with advanced color correction, filtering, rotoscoping, morphing and painting
capacities. Creative Toonz debuted a 2d animation package that automated the more tedious tasks involved in cel animation. Softimage then opened its product line to third-party developers a key step towards the integration from an operating systems point-of-view of the by now fast emerging computer-graphic-to-film image processing system. Softimage and Mental Images announced a rendering technology agreement and released an upgraded 3d package featuring: file management, metaclay, clusters, flock animation, weighted envelopes, channels all based on an “open system” policy and third-party developers.
In 1993 Steven Spielberg chose Industrial Light & Magic to provide the visual effects for Jurassic Park and another major tipping point in media production based on computer-graphics was passed. With computers now operating in networked formats by way of interconnection to servers and transmission systems and able to ingest and output both audio signals and cinema grade high-resolution imagery the so-called “2k production-pipeline”
came into existence. Softimage deployed digital studio (DS) integrating the 3-to-2d software environment around the 2k processing formats then stabilizing within the computer-graphics sector in general. DS supported most elements of the 2 and 3d environments including expressions, dopesheet, ghost mode and shape interpolation.
The push toward “open systems” ironically took another giant leap forward when to exploit the power of the Intel‟s Pentium processor Softimage by then owned by Microsoft deployed the first high-end modeling and rendering package able to run under Irix and Windows NT. Softimage also upgraded to NURBS, relational modeling, trimming, instantiation, polygon reduction, tangent-to-path constraint, Q-stretch, expressions, motion control, actor, particle, mental ray rendering and metaclay. So the palette and toolkit available to those assembling the virtual realities within which the scenes were played continued to expand with each successive leading edge production.
In 1994 Softimage merged with the Microsoft Corporation and in 1995 Wavefront Technologies, Silicon Graphics and Alias Research also entered into a merger agreement activities stimulated by the dot.com investment boom then reaching a fevered pitch. Both company's missions were to focus on developing the world's most advanced 2 and 3d tools for the creation of “digital-content.” At this point the 2k-production system after
about ten years of evolution broke into the corporate exploitation stages of
the process resulting in the “capitalization” of the computer-graphics
industry and its partial vertical integration into either the hard or software sides of the IT sector in the US. At that point the elements of the film industry, in the US, involved in computer-graphics and special effects like ILM, Pixar or DreamWorks, or in Canada, Core, Mainframe, Rainmaker Digital, Toybox West, AlphaCine and Eyes Post Group emerged as the leading players in what was fast becoming the media related components of the enhanced-computer-services sector.
The 2k software environment stabilized the “workflows” moving within the production “pipeline” and enabled a downscale move out of the very upscale film industry into the more backward broadcast and DVD sectors assisted by the relatively low bandwidth requirements of even HDTV television signals. 2k outputs play really well on even very large video based display units. So as the cash flows generated by DVD‟s, videogames on a world wide basis -- surpassed -- those of the still analog and non-interactive components of the sector such as the exhibition and broadcast sectors. The dominant components of the film industry reached yet another critical tipping point in terms of the overall transition towards a digital environment within productive components of the media sector on a continental basis.
But the 2k pipeline still wasn‟t up to speed in terms of being able to service the image-processing requirements for screens of 36 feet and up as was the standard within the exhibition sector. True cinema grade photo-realistic image processing can squeak by with 2k but actually 4-to-8 or better yet 10k is really necessary to play on the larger screens deployed in the exhibition sector. As to fill a 36 foot, let alone a 60 x 80 foot Imax sized screen with clean photo-realistic electronic imagery requires more that 2k. So by the mid 90‟s the 2k production pipeline was already operating with 2k proxies “front-ending” a 4k processing-system based on assembling on a file
server a so-called digital intermediate (DI) able to generate the 2k outputs necessary to television signals, as well as, the 4k and up formats necessary to the large screen based components of the marketplace.
Servers and Network Infrastructure
In 1996 SGI assembled the first server-based “non-linear” editing
network by attaching eight Avid workstations to a central file-server able to ingest, distribute and play back-to-air up to HDTV compatible video signals or broadcast quality 2k computer-graphics. This was merely the application
of the 2k-production pipeline to the broadcast sector. Operational versions of this first video compatible electronic image processing network were installed at CNN‟s Financial and Headline News Networks as well as other high profile broadcast newsrooms in both the US and Western Europe. And the corporate exploitation of the 2k-production environment by the broadcast sector, a process greatly accelerated by the dot.com investment boom, picked up speed on a world wide basis.
Also in 1996 Softimage DS became the world‟s most comprehensive
non-linear production system for creating, editing and finishing films, television programmes and video games. The systems open architecture enabled users to seamlessly integrate picture and audio-editing, compositing, painting, image treatment, special effects, character generation and project management all into one 2 and 3d operational environment. Digital audio editing and non-compressed images met painting, compositing, titling, image treatment and special effects. Softimage then owned by Microsoft not surprisingly then offered a high-end non-compressed post-production system on Windows NT.
Jurassic Park‟s non-animatronic dinosaurs were assembled on SGI
workstations costing tens of thousands of dollars. But due to the high cost, only large facilities with deep pockets or connections to major technology providers could afford what at the time were the apex creative workstations. However this began to change with the migration of the key 2 and 3d applications to the environment provided by Intel‟s Pentium processor and Windows NT. So that by the late 90s a completely new set of visual effects and animation houses emerged based on Intel's Pentium processor. While at the same time the always insightful Steve Jobs moved Apple aggressively into the growing marketplace for image processing workstations based on mass produced and thus cheaper consumer technologies. So while there are still effects facilities working on a variety of platforms, SGI included, most have moved onto either Mac or PC based workstations as these mass produced consumer-electronics are much cheaper than the specialized mini-computers specifically designed for computer-graphics such as SGI.
In 1998 Avid Technology acquired Softimage from Microsoft and "Sumatra" the world‟s first nonlinear animation system was introduced. While Alias Wavefront also introduced a new 3d flagship product branded as Maya Unlimited incorporating Maya Cloth, Fur, Live, and Power Modeling. The upgraded software environment was used by ILM in Star
Wars: Episode I and ILM also invested in Maya “seats” for all their technical directors, designers and computer-graphic artists working in their production department. Also in 1998 Advanced Visualizer was
acknowledged by the Academy as the first commercial software package for modeling, animating and rendering adopted into widespread use to create digital images with sufficient quality for film industry. PowerAnimator became the benchmark for modeling tools with a major influence on visual effects, animation and the video game industries in general.
Discreet Logic also based in Montreal was established in its present format in 1999 when Autodesk merged it with Kinetix another important
American developer of CAD software. Autodesk was already a major source of computer-aided-design and digital content creation systems that “enable our customers to drive their particular production operations forward by way of improving their power to design”. Discreet‟s range of cost-effective
systems and software for digital media creation, management and delivery also crossed all disciplines and platforms from visual effects and editing to animation, game development, web/interactive design, and computer-aided design in general. Discreet is a division of Autodesk one of the world's leading suppliers of digital media creation and management systems with six million customers in 160 countries.
By now the corporate exploitation stage within the computer-graphics revolution was well underway and the providers of the main technology groups, workstations, servers and network infrastructure were organized in varying degrees of vertical or horizontal integration. Meanwhile within the broadcast sector the server-based approach to radio and television production was spreading. For instance in 2001 Sveriges Television selected an SGI system for the Swedish public broadcasting company's news-production facility in Stockholm. The facility was designed around a video-server functioning as a central file storage system and acting as a hub feeding servers located in the other operational components of the network. SVT quickly expanded the network to include it‟s news, sports and current-
affairs departments. As well as being extended to 10 news rooms in Stockholm and 10 regional production centers all inter-connected to the network by either fibre-optic or microwave transmission systems.
The server based approach to radio and television production involves equipping each editing room with a PC-based non-linear editing system or image and audio processing workstation allowing the import and export of