Reconstructing Reality in a Digital Sandbox
by Charles A. Fox, Senior Director of Technology Services
Introduction:
The legal landscape is ever evolving and with it, the tools we use to present our cases. Buzz Lightyear’s iconic phrase, “To infinity and beyond!” may belong to the world of animation, but it also metaphorically represents the bounds we are pushing in legal visualizations. In our courtrooms, 3D animations have transitioned from novel to necessity, playing a pivotal role in elucidating complex facts for juries and judges in many cases. And this is not due to their entertainment value. Animations built on a solid foundation of data can accurately depict events in detailed ways not accessible via photos, security camera footage, or other forms of recording.
Advancements in technology have transformed these animations from illustrative to evidentiary. The 3D scans we employ today offer dimensional precision that allows technologists to build large scenes with millimeter precision. The proliferation of digital recordings from various sources—security cameras, dashcams, and smartphones—allows us to integrate this data into a 3D framework, crafting immersive and reliable reconstructions of events.
Modern 3D animation technology empowers the creation of scaled virtual worlds, testing grounds for hypotheses against the tangible reality of an incident site. This is not merely a theoretical exercise; it's a rigorous investigative process upheld by legal and technical experts to dissect and understand the dynamics of an event.
Gone are the days when 3D models were based on a handful of measurements and a dose of creativity. Today, we harness laser scanners and drones that capture millions of points that are used to define a precise 3D model of an accident scene. Accurately combined scene and object models are then used in an active recreation of an incident, incorporating the crucial element of time. These scenes allow for the analysis of vehicle paths, human movement, and operational machinery and can be further enhanced with environmental factors such as lighting and weather, crucial for understanding visibility and other conditions pertinent to an event.
Developing Visualizations in the Digital Sandbox:
“Visualization” is a broad term that encompasses the result of these investigatory tools. The digital sandbox is a real-time version of the data as it is accumulated. Using a computer, stakeholders interact with the relevant data points like a meticulous child would construct a scene in a sandbox. This allows stakeholders to layer in relevant information anchored by scene geometry and motion. Deliverables from the digital sandbox are often animations simply because the animation is portable, unchangeable, and is easily shared with other parties.
Phase 1: Data Gathering - Capturing the Scene
As an attorney, you know that reconstructing an event for the courtroom isn't just beneficial—it's often critical. The starting point is data collection that paints a detailed and accurate 3D picture of the incident's environment. Whether it's United States Geological Survey (USGS) data of landscapes, drone-captured images of an intersection, or laser scans of vehicles, each dataset is integral to understanding the environment and objects that led to a loss.
- Scene Geometry: Created from various sources like USGS terrain data, survey data, and photogrammetry from drones or laser scanners such as FARO Focus or Leica RTC360.
- Object Geometry: For vehicles or machinery, we might use manufacturer CAD data if available or 3D models verified with laser scanners.
- Motion: Every object that moves in a scene does so in a specific way on a precise timeline. Video can be the gold standard when you know how to reintegrate it into a 3D environment and ensure accurate timing.
Incident data is perishable and immediate scanning of the scene ensures evidence preservation however, this is frequently not possible. We often rely on documentation like first responder photos and video recordings from security cameras that capture information at or near the time of the incident. This photographic and video evidence is invaluable when combined with scan data to reconstruct a virtual scene as it was at the time of the accident even when the laser scan data is captured well after the time of the event.
Phase 2: Analysis
Entering the Sandbox in Real-Time
A digital sandbox doesn't just show a scenario—it immerses you in it. Whether through a head-mounted display or on a computer screen, you're transported to the heart of the incident, able to view the scene from multiple angles, including the perspectives of involved parties. This virtual environment lets you assess the feasibility of witness statements, for example, examining potential obstructions to an eyewitness’s point of view.
Videogrammetry is used to align video within a 3D scene model and enables precise movement recreation. Software like HVE and PC Crash simulate ground vehicle motion and radar and GPS data are useful for aviation accidents. Video and simulation data ensure that by the time we reach trial, the animations presented are founded on accurate and verifiable data.
Critical data analysis is important for developing strong visuals and aligning your legal team. Thorough data collection significantly streamlines this process while taking shortcuts may lead to increased costs and delays. It’s important to secure relevant evidence early that will withstand judicial scrutiny and effectively support your case.
Case Overview: Landscaping Incident
A critical workplace injury occurred at a landscaping site. The individual sustained injuries after being crushed between a motorized concrete transport buggy and a dump truck's tailgate (Figure 1). The landscaper was engaged in the transfer of concrete debris using a skid loader that was stationed on the side of a roadway to load the demolished concrete into a dump truck. During the operational cycle, the buggy operator reversed the concrete buggy away from the loader and into the back of the dump truck, resulting in his upper torso pressing against the vehicle's tailgate. He was trapped in a position that caused his chest to press the safety lever that, when released, stopped the drive system on the buggy.
The reconstruction of the event was informed by two photographs secured by first responders, which became the crux of the spatial analysis (Figure 2). Because the laser scan was taken long after the accident, placing the scale vehicle models accurately within the scene was a challenge. Advanced laser scanning techniques and 3D models of the skid loader, truck, and buggy, augmented by the injured worker’s biometric profile, were utilized to reconstruct the scene. The precise positioning of the skid loader and dump truck, critical to understanding the dynamics of the accident, was extrapolated from the photographs, defining the operational environment that was unsuitably confined.
The 3D model also served as a blueprint for a physical model (Figure 3). This model was instrumental in physical testing to challenge the alleged failure of the buggy's safety mechanism. The tests illustrated the continuous engagement of the drive system when controlled force was applied using a physical surrogate, mimicking the incident conditions.
Case Overview: Urban Transit Incident Analysis
Video camera footage can be informative. Knowing the movements of vehicles, pedestrians, and other objects may explain why an accident occurred. However, it’s frequently not a good tool for determining visibility in nighttime conditions. Video cameras “see” differently in the dark than the human eye can. When a city bus struck and killed a pedestrian, a nighttime conspicuity analysis proved crucial to understanding the bus driver’s view.
The initial step in the reconstruction process involved capturing detailed survey quality data of the accident location using laser scanning to develop a precise 3D model. Concurrently, a detailed lighting analysis was executed, measuring luminosity from multiple sources which were then integrated into the 3D environmental model. This integration was critical in replicating the visual conditions during the incident and included stationary lights, like streetlights, and moving lights on vehicles (Figure 4).
Security camera footage from the bus supplied vital information regarding vehicle movements and pedestrian activity at the time of the accident. A visualization specialist utilized this data to accurately position light sources and animate the scene, enhancing the fidelity of the reconstruction.
Because the bus had a camera viewing the driver, a human factors expert's analysis contributed to the creation of a simulated driver's viewpoint. This virtual driver’s vantage point was critical in showing the driver's potential visual perception during the incident (Figure 5), helping the expert explain why the pedestrian was difficult to see in the background of the headlight glare from oncoming vehicles.
Phase 3: Decision Making
The integration of rapid data collection technologies into legal practice has revolutionized how digital environments are constructed and utilized. These digital sandboxes serve as dynamic tools where layers of information about an event are assembled in valuable ways. Sandboxes allow for the iteration of scenarios, advancing the investigative process by prompting discovery questions. What other useful data exists or could be collected? What should be asked in a deposition? Things learned in the sandbox also guide physical testing to eliminate uncertainties, such as the effect of sunlight on a dirty windshield's transparency.
Once the team has exhaustively analyzed and combined all available information, the reconstructed digital model becomes an educational platform. It translates complex scenarios into visual narratives—images and movies—that animate the event for those outside the investigative circle. These visual aids are instrumental in resolving cases, with the reconstructed reality playing an important role from early in the investigation until the dispute is resolved.
Digital sandboxes play a critical role in helping clients understand and evaluate evidence, which in turn informs their decision-making. These tools allow stakeholders, who may not be well-versed in legal details to better grasp the case strategy and determine the right course of action regarding settlements or trial proceedings.
Deciding to go to Trial:
If the case does not settle and the parties move forward to trial, visual storytelling can significantly enhance juror comprehension of complex information, tapping into the innate human proficiency for processing visual data. It can demystify expert testimony, replacing jargon with clear, impactful imagery. It’s important to remember that jurors will filter the story through their personal experiences, which can both aid in understanding familiar concepts and elicit skepticism towards information that conflicts with their beliefs. Furthermore, a coherent narrative is key; jurors will scrutinize any omissions or inconsistencies with a critical eye, as evidenced by the research of Pennington and Hastie (1992)1. Therefore, visual aids are most compelling when they are part of a well-orchestrated narrative, enhancing and reinforcing expert testimony, especially when prepared in a digital sandbox.
For attorneys incorporating visual aids alongside expert testimonies, a comprehensive understanding of the trial environment is essential. Crafting a strategy that dovetails with the courtroom's technical capabilities is the responsibility of the attorney who brings the expert witness. Legal teams must address several key logistical questions:
- What technology is available at the trial location?
- Can you conduct a preliminary test of this technology to preempt trial hiccups?
- Does the courtroom offer internet connectivity?
- What contingency plans are in place for unforeseen technical glitches?
- What is the judge's stance on the use of technology during proceedings?
Finally, make sure to consider the merits and drawbacks of advanced versus basic technological approaches (for example, equipping jurors with VR headsets versus displaying a video or live feed of the expert's VR demonstration on a screen).
Involving your visual expert early and developing themes in a digital sandbox environment can enhance the quality of visuals your team decides to bring to the courtroom. Such proactive collaboration ensures that the presentation is well-founded and informative, enabling the legal team and sponsoring expert to effectively use the visual products to communicate with the jury.
_______________
1Pennington, N. and Hastie, R. (1992). Explaining the Evidence: Tests of the Story Model for Juror Decision Making. Journal of Personality and Social Psychology, Vol. 62 (2), 189-206. DOI:10.1037/0022-3514.62.2.189
