Response to NIJs A Methodology for Evaluating Geographic

By Clifford Mcdonald,2014-06-17 09:35
21 views 0
Response to NIJs A Methodology for Evaluating Geographic ...

    An Evaluation of NIJ’s Evaluation Methodology for Geographic Profiling Software

    D. Kim Rossmo

    Research Professor and Director

    Center for Geospatial Intelligence and Investigation

    Department of Criminal Justice

    Texas State University

    March 9, 2005

Executive Summary

    This is a response to the National Institute of Justice‟s A Methodology for Evaluating Geographic Profiling Software: Final Report (Rich & Shively, 2004). The report contains certain errors, the most critical of which involves suggested performance measurements. Output

    accuracy is the single most important criterion for evaluating geographic profiling software. The

    report discusses various performance measures; unfortunately only one of these (hit score

    percentage/search cost) accurately captures how police investigations actually use geographic

    profiling. This response addresses the various problems associated with the other measures.

    Geographic profiling evaluation methodologies must respect the limitations and assumptions

    underlying geographic profiling, and accurately measure the actual function of a geographic

    profile. Geographic profiling assumes: (1) the case involves a series of at least five crimes; (2)

    the offender has a single stable anchor point; (3) the offender is using an appropriate hunting

    method; and (4) the target backcloth is reasonably uniform. Additionally, for various theoretical

    and methodological reasons, not all crime locations in a given series can be used in the analysis.

    The most appropriate measure of geographic profiling performance is hit score

    percentage/search cost. It is the ratio of the area searched (following the geographic profiling

    prioritization) before the offender‟s base is found, to the total hunting area; the smaller this ratio, the better the geoprofile‟s focus. There are no intrinsic disadvantages to this measure.

    The other evaluation measures discussed in the NIJ report are all linked to the problematic

    “error distance.” “Top profile area is the ratio of the total area of the top profile region (which

    is not defined) to the total search area. It is not a measure by itself. “Profile error distance” is

    the distance from the offender‟s base to the nearest point in the top profile region (undefined).

    “Profile accuracy” indicates whether the offender‟s base is within the “top profile area”

    (undefined); it fundamental misrepresents the prioritization nature of geographic profiling.

    Error distance is the distance from the offender‟s actual to predicted base of operations.

    While it is easily applied to centrographic measures, the complex probability surfaces produced

    by geographic profiling software must be reduced to a single (usually the highest) point. Several

    researchers have unfortunately adopted this technique because of its simplicity. There are three

    major problems with error distance. First, it is linear, while the actual error is nonlinear. Area,

    rather than distance, is the relevant and required measure. Population (and therefore suspects)

    increases with area size, which is a function of the square of the radius (error distance). The

    second problem with error distance is that it is not a standardized measure because of its

    sensitivity to scale. The third and most serious analytic problem with error distance is that it fails

    to capture how geographic profiling software actually works. Criminal hunting algorithms

    produce probability surfaces that outline an optimal search strategy. As an offender‟s search is

    rarely uniformly concentric, simplifying a geoprofile to a single point from which to base an

    error distance is invalid. The use of error distance ignores most of the output from geographic

    profiling software and undermines the very mechanics of how the process functions.

    A more comprehensive approach to evaluating geographic profiling as an investigative

    methodology needs to consider applicability and utility, as well as performance. Applicability

    refers to how often geographic profiling is an appropriate investigative methodology. Utility

    refers to how useful or helpful geographic profiling is in a police investigation.

    To evaluate geographic profiling properly requires analysing only those cases and crimes

    appropriate for the technique, and measuring performance by mathematically sound methods.

    Hit score percentage/search cost is the only measure that meets NIJ‟s standard of a “fair and rigorous methodology for evaluating geographic profiling software.”



    In January 2005, the National Institute of Justice (NIJ) released A Methodology for

    Evaluating Geographic Profiling Software: Final Report (Rich & Shively, 2004). While the intent of this document is laudable, it is necessary to respond to certain significant errors that are

    contained in the report. Some of these may be the result of the advisory expert panel not

    including professional geographic profilers (defined as police personnel whose full-time function

    involves geographic profiling), “customers” of geographic profiling (police investigators), or

    developers of geographic profiling software. A crime analyst (trained in geographic profiling

    analysis for property crime) was the sole law enforcement practitioner on the advisory panel.

    The most critical error in the NIJ report involves suggested performance measurements.

    The expert panel correctly concluded that output accuracy “the extent to which each software application accurately predicts the offender‟s „base of operations‟” (p. 14) – is the single most

    important criterion for evaluating geographic profiling software (p. 15). The report discusses

    various performance measures, providing short definitions, advantages, and disadvantages (p.

    16). Only one of these measures (hit score percentage/search cost), however, accurately captures

    how police investigations actually use geographic profiling. This response addresses the various

    problems associated with the other measures.


    Geographic profiling is a criminal investigative methodology that analyzes the locations of a

    connected crime series to determine the most probable area of offender residence. It is primarily

    used as a suspect prioritization and information management tool (Rossmo, 1992a, 2000).

    Geographic profiling was developed at Simon Fraser University‟s School of Criminology, and

    first implemented in a law enforcement agency, the Vancouver Police Department, in 1995.


    Geographic profiling embraces a theory-based framework rooted in environmental

    criminology. Crime pattern (Brantingham & Brantingham, 1981, 1984, 1993), routine activity

    (Cohen & Felson, 1979; Felson, 2002), and rational choice (Clarke & Felson, 1993; Cornish &

    Clarke, 1986) theories provide the major foundations. While there are several techniques used

    by geographic profilers, the main tool is the Rigel software program, built in 1996 around the

    Criminal Geographic Targeting (CGT) algorithm developed at SFU in 1991.

    After discussions in the mid-1990s with senior police executives and managers of the

    Vancouver Police Department (VPD) and the Royal Canadian Mounted Police (RCMP), it was

    concluded that several components would be necessary for the successful implementation of

    geographic profiling within the policing profession. These included:

    ? creating personnel selection, training. and testing standards;

    ? following mentoring and monitoring practices;

    ? developing usable and functional software;

    ? establishing case policies and procedures;

    ? identifying supporting investigative strategies;

    1 Examples of the use of spatial statistics for investigative support can be identified in the

    literature as far back as 1977. During the Hillside Stranglers investigation, the Los Angeles Police

     2 Department analyzed where the victims were abducted, their bodies dumped, and the distances between

    these locations, correctly identifying the area of offender residence (Gates & Shah, 1992). None of these

    isolated applications, however, led to sustained police use or organizational program implementation.

    ? building awareness and knowledge in the customer (police investigator) community; and

    ? committing to evaluation, research, and improvement.

    Over the course of the next few years these components were developed, first for major

    crime investigation, and then for property crime investigation. Personnel from various

    international police agencies were trained in geographic profiling. Their agencies signed

    memoranda of understanding agreeing to follow the established protocols, and to assist other

    police agencies needing investigative support. Training standards for geographic profilers were

    eventually adopted by the International Criminal Investigative Analysis Fellowship (ICIAF), an

    independent professional organization first started by the Federal Bureau of Investigation (FBI)

    in the 1980s.

    For a geoprofile to be more than just a map, it must be integrated with specific strategies

    investigators can use. Examples of strategies identified for geographic profiling include: (1)

    suspect and tip prioritization; (2) database searches (e.g., police information systems, sex

    offender registries, motor vehicle registrations, etc.); (3) patrol saturation and surveillance; (4)

    neighborhood canvasses; (5) information mail outs; and (6) DNA dragnets. The level of

    resources required by these strategies is directly related to the size of the geographic area in

    which they are conducted.

    While not used by professional geographic profilers, there are two derivative geographic

    profiling tools also mentioned in the NIJ report: NIJ‟s own CrimeStat JTC (journey-to-crime)

    module; and The University of Liverpool‟s Dragnet. Both of these systems were developed in

    1998. Neither is a commercial product, and training in their use, beyond software instruction

    manuals, is currently unavailable. Little is known about a fourth geographic profiling software

    program, Predator, first mentioned in 1998 on Maurice Godwin‟s investigative psychology


    Geographic Profiling Evaluation

    There are three methods for testing the efficacy of geographic profiling software. The first

    uses Monte Carlo simulation techniques. These test the expected performance of the software on

    various point patterns representative of serial crime sites. The major advantage of this approach

    is the ability to generate large numbers of data cases (e.g., 10,000). The major disadvantage is

    the likelihood the site generation algorithm‟s underlying assumptions do not accurately reflect

    the geographic patterns of all serial crime cases. In addition, the additional information

    associated with an actual case that can help refine a geoprofile is not present.

    The second and most common method of evaluating geographic profiling software

    performance involves examining solved cases. This technique has been used by Rossmo (1995a,

    2000), Canter, Coffey, Huntley, and Missen (2000), Levine (2002), Snook, Taylor, and Bennell

    (2004), and Paulsen (2004). The major advantage of research using historical (cold) cases is that

    with sufficient effort a reasonably sized sample of cases can be collected. Disadvantages include

    sampling bias problems and the need for extensive data review.

    The third method tracks geographic profiling performance in unsolved criminal

    investigations. This approach is the best of the three as it measures actual not simulated

    performance under field conditions (Rossmo, 2001). It also serves as a blind test as the “answer”

    is not known at the time of the analysis. Monitoring actual case performance is slow, however,

    as it is necessary for a case to be solved before it can be included in the data sample.

    Every trained geographic profiler is required to keep a case file that records the details of

    their work. The log includes fields for case number, sequential number, date, crime type, city,


region, law enforcement agency, investigator, number of crimes, number of locations, type of

    analysis, report file name, case status, and result (when solved). This file has both administrative

    and research purposes. It was encouraging to see the NIJ report recommend the use of logs and

    journals by individuals involved in geographic profiling. However, considering how much there

    is to learn with any new police technology (especially in regards to investigator utility versus

    software performance), it seems more prudent for all users, and not just a sample, to keep

    detailed records.

    Geographic Profiling Evaluation Methodology Evaluation Premises

    NIJ‟s purpose was “to develop a fair and rigorous methodology for evaluating geographic

    profiling software” (p. 4), and their report identifies law enforcement officials as the key

    audience for the evaluation. With this in mind, the following premises are used as the basis for

    the discussion in this response.

    Any geographic profiling evaluation methodology should:

    ? follow the limitations and assumptions underlying geographic profiling;

    ? analyze exactly what the geographic profiling software produces, and not a simplification

    or generalization of its output;

    ? measure, as accurately as reasonably possible, the actual function of a geographic profile;

    ? use the highest level measurements possible (i.e., ratio/interval/ordinal/nominal); and

    ? be based on validity and reliability concerns (and not on tangential factors such as “it is

    easier,” “it has been done that way before,” or “the software has limitations).

    It is tempting, in the effort to increase a study‟s sample size, to collect cases from large

    databases derived from records management systems (RMS). However, if the details of the

    crimes are overlooked, inappropriate series will be included: GIGO garbage in, garbage out.

    Wilpen Gorr, Michael Maltz, and John Markovic have warned us of the importance of data

    integrity and specificity issues. “You really need to know the capacities and limitations of this less then perfect [crime] data before you dump it into a model” (John Markovic, International

    Association of Chiefs of Police, NIJ CrimeMap listserve, January 31, 2005).

    To prepare a geographic profile properly involves first making sure the case does not violate

    any underlying assumptions. Furthermore, only those crime locations in the series that meet

    certain criteria can be used in the analysis. This is one of the reasons why a geoprofile requires

    anywhere from half a day for a property crime case to up to two weeks for a serial murder case.

    A significant portion of the geographic profiling training program is spent learning to understand

    these issues so the methodology is not improperly applied. These complexities are why testing,

    monitoring and mentoring, and review exist.

Geographic Profiling Assumptions

    Any algorithm or mathematical function is only a model of the real world. The

    appropriateness and applicability of weather forecasting techniques, multiple linear regression,

    the spatial mean, or horserace odds are all premised on various assumptions. If those

    assumptions are violated, or if the processes of interest are not accurately replicated, the model

    has little value. Using atheoretical algorithms for police problems is tantamount to fast food

    crime analysis.

    There are four major theoretical and methodological assumptions required for geographic

    profiling (Rossmo, 2000):


    2 over the time period of the crimes. 1. The case involves a series of at least five crimes, committed by the same offender. The 3. The offender is using an appropriate hunting method. series should be relatively complete, and any missing crimes should not be spatially 4. The target backcloth is reasonably uniform. biased (such as might occur with a non-reporting police jurisdiction). Geographic profiling is fundamentally a probabilistic form of point pattern analysis. Every 2. The offender has a single stable anchor pointadditional point (i.e., offense location) in a crime series adds information, and results in greater

    precision. A minimum of five crime locations is necessary for stable pattern detection and an

    acceptable level of investigative focus; the mean in operational cases has been 14 (Rossmo, 2000,

    2001). Monte Carlo testing shows with only three crimes the expected hit score percentage

    (defined below) is approximately 25%. By comparison, the expected search area drops to 5%

    with 10 crimes.3 The resolution of any method will be poor if tested on series of only a few crimes.

    The NIJ evaluation methodology recommends analyzing cases with as low as three crimes in

    the series. While there may be some research interest in studying performance for small-number

    crime series, the report is supposed to lay out guidelines for evaluation methodologies. The

    document seems at times to be confused as to its role. Research and evaluation are separate

    processes. At a minimum, any research results should be reported separately, and it should be

    made clear they do not represent operational geographic profiling performance.

    For evaluation purposes, it is inappropriate to include cases that fall outside the

    recommended operational parameters. As small-number crime series are easier to obtain than

    large-number crime series, there is a risk they will inappropriately drive the findings. For

    example, the distribution for the number of crimes in Paulsen‟s (2004) analysis was heavily

    skewed to small-number series. Of 150 cases, only 37 (25%) meet the minimum specified

    requirement in geographic profiling of 5 crime locations and 22 of those were on the borderline

    (6-7 crimes). Only 15 cases (10%) involved more than 7 crimes.

    If the offender is nomadic or transient then there may not be a residence to locate. If the

    offender is constantly moving residence, then multiple anchor points could be involved in a

    single crime series, confusing the analysis, and possibly resulting in a violation of the first

    assumption. It should also be remembered that what constitutes a residence for a street criminal

    might vary from middle-class expectations. Two geoprofiled burglary cases illustrate this point.

    In the first, the “home” for a group of transient gypsies was a motel where they temporarily

    stayed while they committed their crimes. In the second, the homeless offender‟s base was a

    bush in a vacant lot where he slept at night. It is important in geographic profiling to consider

    the details of the case, the timing of the crimes, and the nature of the area where the peak

    geoprofile is located. Like all investigative tools, it should be used intelligently. See the

    discussion below regarding applicability, performance, and utility, in the section General

    Methodological Comments.

    2 The offender‟s anchor point appeared to be their residence in the majority (about 85%) of the

    cases analyzed by professional geographic profilers. Other examples of anchor points include the

    offender‟s workplace or (for students) school, past residence (if the move was recent), and surrogate

    residences (homes of family members and friends where the offender actually lives).

    3 The benchmark is 50%, what you would expect from a uniform (i.e., non-prioritized) search.


    Offender hunting method is defined as the search for, and attack on, a victim or target

    (Rossmo, 1997, 2000). Geographic profiling is inappropriate for certain search and attack

    methods. The residence of a poacher (an offender commuting into an area to commit crimes) by

    definition will not be within the hunting area of the crimes (though he may be using some other

    anchor point, such as his workplace or a “fishing hole”). The NIJ methodology suggests that “commuters” be included in any evaluations. How that is to be done, however, is not made clear, as the report acknowledges CrimeStat and Dragnet are unable to handle this type of offender

    (nor can Rigel, as this is an assumption violation). Gorr (2004) presents some interesting and

    useful ideas for expanding geographic profiling systems to such cases and these should be

    explored (perhaps with the addition of a directionality component). As discussed above, it is

    important to distinguish research from evaluation, and report the results of each separately.

    While most burglars identify targets during their routine activities in areas of familiarity,

    others watch for news of estate sales or use accomplices who read luggage nametags at airports.

    Stalkers (offenders who do not attack victims upon encounter) are also problematic. In one case

    example, the offenders in a series of armed robberies of elderly victims in Los Angeles went to

    hospitals and shopping malls, selected suitable victims, and then followed them home where the

    robbery occurred. In this situation, the victims were choosing the crime sites not the offenders.

    A geographic profile based on the robbery locations would therefore be wrong. Instead, the

    victim encounter sites (the shopping malls and hospitals) should be used because these are the

    locations the robbers had control over.

    All criminal profiling involves the inference of offender characteristics from offense

    characteristics. But this assumes freedom of offender choice; the more constrained the behavior,

    the less valid the inferences. A uniform (isotropic) target backcloth is thus a necessary

    assumption for geographic profiling. Certain offenders hunt victims whose spatial opportunity

    structure is patchy. A predator attacking prostitutes is one such example; he has little freedom of

    spatial choice as he must hunt in red light districts where potential victims are located.

    A final caution is necessary regarding what a geographic profile actually produces. It shows

    the most likely area for an offender‟s anchor point (“base of operations”). While this is most

    often their residence, in other cases it may be their former residence, workplace, a freeway

    access point, or other significant activity site. This underlines why it is inappropriate to analyze

    cases without consideration of the underlying theory and environmental background. For

    example, the geoprofile in a series of bank robberies that occurred from 12:30 pm to 1:00 pm fell

    onto a commercially zoned section of the city. This led to the correct inference the offender was

    committing the crimes during his lunch break; while the geoprofile said nothing about where he

    lived, it accurately pointed to where he worked.

    For testing purposes, the use of numerous anchor points as prediction sites can quickly

    produce tautological results. Absent significant a priori information (as in the above bank

    robbery case), it represents multiple “kicks at the can,” and performance results need to be

    statistically adjusted accordingly.

Crime Site Selection


    4 should be

    used in the analysis. Independence of site selection is one issue. An arsonist walks down the

    street, sets fire to a dumpster, then walks around the corner and sets fire to another dumpster. The selection of which crimes to use in a geoprofile is an important process called scenario We have two crimes, two addresses, two times, and potentially two victims. However, for creation. For various theoretical and methodological reasons, not all crime locationsgeographic profiling purposes we would eliminate the second crime location, as it is not

    independent of the first location.

    A second issue is spatial displacement. If the offender moves his or her area of operations

    because of police saturation patrolling, for instance, it would likely be inappropriate to combine

    post-displacement with pre-displacement crime locations.

    NIJ Performance Measures

    The NIJ report lists several geographic profiling performance measures in a table (p, 16),

    explaining “Individual panelists also presented what they felt was their „favorite‟ output

    measure” (p. 9). Reviewing the transcript in the report‟s appendix shows much debate amongst

    the members of the expert panel on this particular topic. Employing multiple performance

    definitions, unless they all reflect actual use conditions, is confusing and disadvantageous.

    Geographic profiling is an information management strategy, primarily used for suspect and area

    prioritization. The proper way to test its efficacy is to employ a method that simulates what

    happens in the real world. The hit score percentage (or search cost) is a measure that

    satisfactorily accomplishes this goal. Their “popularity” notwithstanding, the other measures

    discussed in the report do not. The problems with these measures are discussed below.

Hit Score/Search Cost

    The NIJ report incorrectly states “there are no existing standard for measuring accuracy” (p. 16). The hit score percentage (also referred to as search cost) is such a standard, one that

    accurately and correctly captures geographic profiling performance. It was established in the

    early 1990s (Rossmo, 1993b), and is used by all professional geographic profilers to quantify

    geoprofile accuracy. The report lists two disadvantages associated with the hit score

    percentage, both of which are carefully scrutinized below. However, let us first closely examine

    what hit score percentage measures.

    The hit score percentage is a measure of geographic profiling search efficiency. It is defined

    as the ratio of the area searched (following the geographic profiling prioritization) before the

    offender‟s base is found, to the total hunting area; the smaller this ratio, the better the geoprofile‟s focus. It is calculated by first adding the number of pixels with a hit score

    (likelihood value) higher than that of the pixel containing the offender‟s residence to half the

    number of “ties” (pixels with the same hit score), and then dividing by the total number of pixels in the hunting area (40,000 for Rigel).

    The hunting area is defined as the rectangular zone oriented along the street grid containing

    all crime locations. These locations may be victim encounter points, murder scenes, body

    dump sites, or some combination thereof. The term hunting area is therefore used broadly in

4 There are other issues for multiple location crimes but these more typically occur in violent

    sexual offenses. A rape, for example, may involve separate encounter, attack, assault, and victim release

    sites (Rossmo, Davies, & Patrick, 2004).


    the sense of the geographic region within which the offender chose after some form of

    search or hunting process a series of places for criminal action. Locations unknown to

    authorities, including those where the offender searched for victims or dump sites but was

    unsuccessful or chose not to act, are obviously not included…. A priori, we do not know

    where this hunting area is we only know the locations of the reported, and connected,

    crimes. Technically, a geoprofile stretches to infinity; the hunting area is only a

    standardized method of displaying results so that important information is shown, and

    unimportant information is not. (Rossmo, 2000)

    In the nomenclature of point pattern analysis, the crimes are events and the hunting area is the

    study region (Gatrell, Bailey, Diggle, & Rowlingson, 1996).

    Two factors influence the hit score. First, the resolution of the pixel grid slightly influences

    precision as more pixels result in more decimal places (e.g., 4.87% vs. 4.9%). Second, the size

    of the hunting determines the denominator of this ratio. For example, a larger hunting area 5 Dragnet also employs a rectangular search area, defined by the offenses but produces a larger denominator, therefore reducing the hit score percentage or search cost. Rigel magnified by 20%, displayed in a grid of 3,300 pixels (Canter et al., 2000). uses a minimal bounding rectangle for its hunting area, with a small addition of a guard area for While the NIJ report is correct in stating this measure is dependent upon how the search area edge defined, this is not a disadvantage for the purposes of comparative evaluation. As long as the

    same method is used consistently for all analyses in the comparison, relative performance will be

    the same. When you divide one ratio by another with a common denominator, the denominator

    is cancelled out. Therefore, the hunting area can be defined as a rectangle, circle, convex hull

    polygon, or any other logical shape. As long as the same procedure is consistently used,

    numerator comparisons will be independent of its size.

    The second disadvantage listed in the report for the hit score percentage is a claim that it is

    “Subject to severe changes in output display based on method of thematically mapping the

    output” (p.16). NIJ Mapping and Public Safety (MAPS) personnel were queried to clarify the

    meaning of this. It turns out that this is not an inherent disadvantage of the hit score

    percentage/search cost measure, but rather is a program-specific problem for NIJ‟s CrimeStat

    software. Apparently, its output, when loaded into a geographic information system (GIS), can

    be dramatically altered depending on the thematic classification used. Nevertheless, as

    CrimeStat produces a “hit score map” (p. 5) this issue should be solvable (otherwise, the software could not prioritize a list of suspects or areas the main function of geographic profiling).

    The equivalent of hit score percentage or search cost can also be calculated for first-order

    centrographic statistics (e.g., spatial mean, spatial median, “eyeball” estimates, etc.). A search

    process from a single point estimate involves an outward spiral. The square of the error distance

    divided by the total area covered by the crimes provides the appropriate search performance

    measure. If this is being calculated on a grid, half the “ties” (the number of pixels located exactly the error distance away) must be added to the numerator

    5 Rigel calculates the boundaries of the hunting area by adding half the mean x and y interpoint

    distances to the most eastern and western, and northern and southern crime sites, respectively (Boots &

    Getis, 1988; Gatrell et al., 1996). The geoprofile is generated within the hunting area.


    Previous Evaluations of Geographic Profiling Performance

    Three reported studies have examined geographic profiling performance using hit score

    percentage or search cost. In research on 13 solved serial murder cases from the United States,

    Canada, and Great Britain, involving 15 serial murderers, 178 victims, and 347 crime locations,

    Rossmo (1995a) observed the CGT algorithm located the offender‟s residence in the top 6% of

    the hunting area.

    Canter et al. (2000) found similar results in their study of 70 U.S. serial killers using

    Dragnet. The mean search cost was 11%; 87% of the offenders‟ home bases were found in the

    top 25% of the search area, 51% in the top 5%, and 15% in the top 1%.

    In 2001, a review was conducted of all now-solved (but initially unsolved) cases analyzed

    by full-time police geographic profilers (from the Vancouver Police Department, Royal

    Canadian Mounted Police, Ontario Provincial Police, and British National Crime Faculty). The

    study combined the 39 hot cases with 31 cold cases, for a total of 70 serial crime cases,

    representing 1,426 offenses and 1,726 crime locations (Rossmo, 2001). These involved, in order

    of frequency, murder, rape, arson, robbery, and sexual assault. The hot cases came from police

    agencies in North America, Europe, and Africa, and occurred from 1991 to 2001.

    There was an average of 20 crimes (25 locations), and a median of 14 crimes (17 locations)

    per case. The mean hit score percentage was 4.7% (SD = 4.4%), and the median was 3.0%. The

    performance on hot cases was about 1% better than for the cold cases, likely the result of the

    extra information and analysis time involved.

Top Profile Area

    The top profile area is defined as the ratio of the total area of the top profile region to the

    total search area. It used with profile error distance. This is actually not a distinct performance

    measure (at least as explained in the report). It forms an integral part of “profile error distance”

    and “profile accuracy,” but does not stand alone as a performance measure. Apparently top

    profile area is similar to search cost, but the mathematical relationship is unclear. The “top

    profile region” is not defined in the NIJ report beyond the vague “the predicted most likely region containing the base of operations” (p. 15). Is this the top 10% of the area? The top five

    square miles? The top profile area is apparently a subjective and crude measure.

Profile Error Distance

    This is defined as the distance from the offender‟s base to the nearest point in the top profile

    region. The report claims it takes advantage of the whole profile, but this is incorrect. It only

    takes into account the “top profile area,” which is not defined. The profile error distance cannot distinguish between offender locations within the “top profile area” – they are either in or out of this region. Moreover, for offender locations outside the “top profile area,” we are back to the

    use of a linear measure, with all the problems associated with error distance.

Profile Accuracy

    This is defined as a dichotomous measure of whether the offender‟s base is within the top

    profile area, and provides a simple indication of whether the geoprofile was “correct.” The very

    use of such terms as “correct” or “accurate” shows a fundamental misunderstanding of

    geographic profiling. This is like saying a golf drive from the tee was “wrong” because it only


Report this document

For any questions or suggestions please email