It is said that “Eyesight is more powerful than hearsay” – an adage that proves itself in connection with the collection of information in the modern world. Vision is the source of more than 80% of the information we pick up. 90% of the information in the world has been created over the last two years and the source of 80% of that information were various types of photographs and video images.
The Video Content Analytics (VCA) market, including sales of products, services and upgrades, has been estimated in the last year at about US$ 13.5 billion. According to a study published in the USA in March 2013, this market is expected to generate a sales turnover of about US$ 39 billion by 2020. The data prove that the need for technology is acute and the development process is inevitable.
When this issue is examined more closely and thoroughly, it becomes obvious that the ambitious objective stated above will only be achieved after the numerous challenges that are already visible and may hinder the development of this field have been resolved.
The players involved in this field will be required to provide solutions to a range of legal, ethical and technological problems.
The existing operational needs that have already been identified focus primarily on surveillance/monitoring of territories/areas, denial of passage/crossing, identification of the object’s activity, identity analysis, understanding the operational trend and attempting to analyze its objective. The existing systems are employed in the context of the relevant missions at airports and sea ports, land, sea and air surveillance applications, border surveillance, critical infrastructures, manufacturing facilities, buildings, urban areas, banks, financial institutions, hospitals, shopping centers, sports facilities, logistic centers, traffic routes and so forth.
The availability of video sources has increased significantly over the last few years, and is expected to reach capacities that we would not be able to handle at the required scope and rate without advanced automated tools.
The availability of technological systems that are relevant to the field has improved as well: advanced algorithms, dedicated processing hardware and end equipment were developed, the applications available are effective and help human operators accomplish their missions under conditions of multiple sources and simultaneous occurrences.
The human factor, without VCA applications, could be a substantial hindrance: in 2020, some 3.3 trillion video hours are expected to be collected just from security and surveillance cameras. If we were to analyze just 20% of that material by human operators, we would require 100 million people working 8 hours a day over 300 days per year. The cost of employing skilled operators and building suitable display centers and the physical and mental pressures associated with the attempt to understand the status picture and manage it during stressful periods will make the human factor the weakest link.
The need to utilize advanced automated tools as a solution for the weakness of the human solution and the costs of the errors, which could be critical, will boost the growth trend of the VCA market even further.
Having analyzed the missions and the technological solution trends, I believe it would be appropriate to examine the operational requirements presented to the technological systems according to the distance and range criteria and not just according to the mission.
Long-range VCA: the primary missions for this category are, for example, monitoring of expansive areas, border defense, denial of crossing, and identifying the intruder’s trend, type and mission. These missions are normally characterized by space/time that enable a measured response pursuant to a structured analysis and decision-making process by the defending side.
The requirements derived from the systems match the above: 24-hour detection under any weather conditions, in all three dimensions (air, land and sea). The object should be identified relatively coarsely, along with the heading and relevance of the threat. The primary objective is to achieve situational awareness, a status picture and intelligent early warning.
Medium-range VCA: the primary missions for this category are covering large territories and specific areas, monitoring, analyzing and predicting behavior. The shorter range calls for alertness, a faster response and prediction of the forthcoming activity trend. For example, a group of people assembling in a yard, a person moving inside a warehouse, an object left unattended in a controlled area.
The requirements derived from the systems: detection through smoke, camouflage, rain, and under any lighting conditions, including 3D photography. The analysis capabilities are required to enable comparison with past images and parallel sensors, understanding of object behavior, tracking the object along its route of advance, collection of intelligence and so forth.
Short-range VCA: the short range calls for the most complete identification possible and for immediate response within operational time in the area where the incident occurred as well as in derivative areas or sites, in the context of a scenario most of which is based on automatic activities/sequences, prepared and rehearsed in advance. Classic examples are identification of a quarrel between prison inmates, analysis of social connections between inmates by monitoring them passing each other during outdoors exercise, monitoring of offensive intentions and so forth.
The vision sounds fantastic and all of the technological components required exist and are readily available. We can assume that one day, while we are walking down the street, the sign opposite us will photograph us, identify who we are, understand that our car insurance is about to expire in a week and present us with an attractive offer, custom-tailored to our specific needs (Google does that through AdSense, for example). Alternately, our Google Glass spectacles will photograph the passers-by heading toward us and project their names and details on where we know them from on our display.
All of these capabilities are amazing, instructional and enable applications with a significant business-driven model. The problem lies with the privacy protection legislation that evolves accordingly. The implementation of such capabilities through the Google Glass technology has been banned in the USA recently. The constraints are naturally less severe in enclosed environments that are monitored anyway and possess unique characteristics, like prisons and correctional facilities.
The significance of the above is that personal-level information will have to be collected using complementary tools, on the basis of historic behavior symptoms, limited local databases and so forth, rather than on the basis of large-scale, general biometric databases.
One of the most significant challenges involves the integration of the video sources and types. Pursuant to the development of numerous video sources of various types that the systems are required to integrate and analyze, including such standard video input sources as stationary cameras, smartphones and clips on Youtube, and such image types as radar images, thermal/IR images and 3D images, the abundance of sources, types and standards necessitates the development of new algorithms.
Storage also constitutes a challenge. The amount of devices that generate video has increased dramatically with the growing use of smartphones as a global phenomenon. A typical company, included in the Fortune 500 list, operates hundreds of surveillance/monitoring cameras that require a storage capacity on the scale of several petabytes. The information is normally kept for 30 days and then erased. With the exception of irregular cases, the accumulated information is neither analyzed nor subjected to historic examination, and in fact does not contribute to the company’s security to the extent of its full potential.
The information received from multiple sources simultaneously can be analyzed and examined in real time. The challenge is intensified in view of the privacy protection legislation and the need to protect the information and prevent manipulation using cyber warfare techniques.
Finally, we reach one of the most complex challenges – the human element. On average, a human operator cannot remain fully alert opposite TV/video screens for more than 20 minutes. As time passes, the operator’s ability to analyze and respond within the required interval diminishes. A substantial challenge involves the manner in which the information is displayed to the operator and the integration of automatic and manual operations at the required rate.
An analysis of the operational needs, the technological development trends and the existing challenges, as outlined above, indicates that the establishment of independent VCA systems will not provide the quality standards to which the solution that may be required by the end of the present decade should conform.
In view of the above, it seems that it is already necessary to integrate in systems and projects in this field additional technologies that would make it possible to overcome the constraints outlined above and empower the capabilities and contribution of the VCA systems to the clients, such as Big Data, Deep Learning, Internet of Things, automation & robotics and broadband communications.
The capability inherent in VCA systems embodies a tremendous potential the technological and business utilization of which is only at its outset. VCA systems will empower autonomous and automatic operational capabilities that would make it possible to detect, understand and take action without further burdening the human operator. The application of these systems should include the integration of technologies in complementary and tangential fields, while preparing technologically and with regard to infrastructures at the information core of the organization.
Brig. Gen. (ret.) Israel (Russo) Rom served, among other positions, as Head of the Technologies Administration of the Israel Prison Service. Today he is an entrepreneur and director of programs involving the integration and implementation of state-of-the-art technologies