See What I See: Machines With Mental Muscle
Science Daily (Oct. 20, 2008) ?ª The way we use and interact with machines is undergoing a profound change as computers are programmed to learn from experience and see more how we see. European research into machine learning is pushing back the boundaries of computer capabilities.
Computers do not see things the way we do. Sure, they can manipulate recorded images, for example, but they currently understand very little about what is inside these pictures or videos. All the interpretation work must be done by humans, and that is expensive. But one European project is making computers more similar to us in their ability to interpret images and their surrounds. Individuals from all walks of life, as well as sectors such as industry, services and education, stand to reap immense benefits from semi-autonomous, more intuitive machines that are able to do things which were, until now, either not possible, super expensive or the preserve of humans. This has been made possible thanks to the developments in, and convergence(?á ºÏ;?ÛºÏ)of, methods for creating, obtaining and interpreting metadata ?C at its simplest level this is data about data, or facts about facts ?C in complex multimedia environments. MUSCLE, an EU-funded super project which created a pan-European network of excellence involving more than 30 academic and research institutions from 14 countries, has come up not only with new paradigms ??Àý but a range of practical applications. Vast scale The scale of the project was so vast, a special section to showcase its achievements has been set up in the 3D Second Life internet virtual world, which has millions of denizens(?Ó×?Õß,?ÓÃñ). The Virtual MUSCLE experience inside Second Life has been created as a one-stop information centre to ensure the continuation and sustainability of the project's achievements. Users are impersonated as avatars1 (computer representations of themselves) enabling them to experience multimedia content by literally walking through it. They are able to hold real-time conversations with other members of the community, exchange experiences, or just simply browse. After an initial two years of collaborative research across the MUSCLE network, a series of showcases were established with several institutions working together on each one to produce practical applications. Virtual tongue twister
One of these is an articulatory talking head, developed to help people who have difficulties in pronouncing words and learning
vocabulary. This 'insightful' head models what is happening inside the human mouth, including where the tongue is positioned to make particular sounds, so the users can copy what they see on screen. A second showcase functions as a support system for complex assembly tasks, employing a user-friendly multi-modal interface ?çÃæ. By augmenting the written assembly instructions with audio and visual prompts much more in line with how humans communicate, the system allows users to easily assemble complex devices without having to continually refer to a written instruction manual. In another showcase, researchers have developed multi-modal Audio-Visual Automatic Speech Recognition software which takes its cues from human speech patterns and facial structures to provide more reliable results than using audio or visual techniques in isolation. Similarly, a showcase which has already attracted a lot of publicity, especially in the USA, is one that analyses human emotion using both audio and visual clues. "It was trialled on US election candidates to see if their emotional states actually matched what they were saying and doing, and it was even tried out, visually only of course, on the enigmatic ÈçÃÕµÄ Mona Lisa," says MUSCLE project coordinator Nozha Boujemaa. Horse or strawberry? Giving computers a better idea of what they are seeing or what the inputs mean, another showcase developed a web-based, real-time object categorisation system able to perform searches based on image recognition - photos including horses, say, or strawberries! It can also automatically categorise and index images based on the objects they contain. In an application with anti-piracy potential, one showcase came up with copy detection software. "This is an intelligent video method of detecting and preventing piracy. There is a lot of controversy at the moment about copyright film clips being posted on YouTube and other websites. This software is able to detect copies by spotting any variation from original recordings," Boujemaa explains. "Another application is for broadcasters to be able to detect if video from their archives is being used without royalties been paid or acknowledgement of the source being made. Europe's largest video archive, the French National Audiovisual Archive, has now been able to ascertain that broadcasters are only declaring 70% of the material they are using," she tells ICT Results. Other types of recognition software, effectively helping computers see what we see, can remotely monitor, detect and raise the alarm in a variety of scenarios from forest fires to old or sick people living alone falling over. The latter falls under the heading of "unusual behaviour" which also has applications in video security monitoring with "intelligent" cameras able to alert people in real time if they think somebody is suspicious. "During the course of the project, we produced more than 600 papers for the scientific community, as well as having two books published, one on audiovisual
learning techniques for multimedia and the other on the importance of using multimedia rather than just monomedia," she says. Although the massive project has now wound down, its legacy remains online, in print and most of all in a host of new applications that will affect the lives of people all over the world.
Foundations For The World Wide Grid ?ñ×Ó,ÌúË?Íø ?ñ×Ó,
Science Daily (Nov. 21, 2008) ?ª The dream of using the Internet to allow people to access as much computer processing and storage power as they need, when they need it, is a step closer thanks to European researchers.
Although a World Wide Grid running on top of the Internet is still probably years away from being a reality, the grid, like the web before it, is starting to take shape between academic and scientific institutions. Where the Internet is a communications channel between computers, the grid goes beyond this by not just using the Internet for communications but also as a means of sharing computing resources. Every computer and user can access and make use of the combined resources of the grid. As things stand at the moment there are a series of isolated grids which allow the resources of clusters of computers, at different universities for instance, to be shared. Each of these grids is usually based on its own proprietary Ë?ÓªµÄ middleware ÖÐ?äÉè?? which makes interoperability(ÈÃÈíÓ??þÔÚ?àÖÖÆ?ÅÆ?úÆ?ÉÏÄÜÓÐÒâÒåµÄ?µÍ?)impossible. Middleware is a type of software which connects hardware resources to a grid. There are different middlewares available, each tailored for different scientific, commercial or industrial usage. Grid confined to experts Another barrier to the development of the grid system is its difficulty of use, requiring as it does now knowledge of specialised computer languages and coding skills. It is against this background that the EU-funded g-Elipse project has been developing an easy-to-use, Windows-like graphical interface which allows access to grid resources with a few mouse clicks. Says project coordinator Mathias St??mpert: "Entering and using a grid has been too difficult for most people, so we are developing a system which allows the ordinary student to use grid resources. Until now, these have only been available to academics and scientists able to enter complicated command lines. Instead of something that takes months to learn, we are developing a graphical user interface (GUI) which can be operated by anybody with a basic knowledge of computing." The idea is to develop a system which is middleware independent, so a user can
access any grid in exactly the same way using the same actions and commands on the GUI. The g-Eclipse system also aims to allow users to work with different grids at the same time and switch data between them.
New sort of browser "You can think of g-Eclipse as a browser for what will become the World Wide Grid," says St??mpert. "It searches for and displays the resources that are available, and allows the user to access them. Complicated computing jobs which need more processing or storage than are available on the user's system can be sent to the grid. Data can be transferred from the local computer to the grid and workflows can be managed." The project is making use of the Eclipse open-source ecosystem, which has thousands of developers and a very large user base and is host to numerous application development projects from around the world. "We chose Eclipse as our medium because it allows us to create a user base and it also means anybody in the world can contribute. Eclipse projects are really transparent and open, more so even than Linux, and source code can simply be reused between Eclipse programmes." "With support from the Eclipse Foundation we get a lot of functionality from the ecosystem and the use of a lot of infrastructure, such as software or code repositories. The project develops a life of its own which allows it to continue even after the EU funding has been spent," St??mpert says. Middleware-independent architecture So far, the
middleware-independent g-Eclipse core architecture has been configured to work with two brands of middleware. Initially, it was assessed using the scientific gLite middleware which is used by European scientific institutions, but plug-ins have also been developed for the GRIA middleware which is in commercial and industrial use. The system has also been configured for use with computing clouds, and specifically Amazon.com's Elastic Compute Cloud. Cloud computing allows firms which have installed computer capacity to cope with peak periods, such as Christmas, to hire the excess capacity out. Site administrators expecting unusually heavy traffic can lease tens, hundreds or even thousands of virtual servers from firms like Amazon, for minutes, hours or days at a time as and when the extra capacity is required. While other GUIs have been developed for this purpose, g-Eclipse is currently the only one allowing data to be transferred between the "real" world of grids and the virtual world of clouds. Supporting other developers "We are not just supporting the individual user, although we do already have a lot of new users sending their daily jobs to the grid, but also a framework that can be used for other developers to build their applications on." says St??mpert. His hope is that the Eclipse community, having seen the value of the work to date, will continue to push back the boundaries with other developers plugging g-Eclipse into all the grids and clouds which connect to the Internet.
"While at this early stage our users are mainly students, a few years down the road g-Eclipse could be a part of everybody's desktop. Perhaps there will be a layer in computer operating systems which allows
applications to be executed on the grid rather than the local desktop." If and when that happens, every PC user could well have access to all of the computing power and speed they could possibly require. g-Eclipse was funded by the ICT strand of the EU's Sixth Framework Programme for research.