|

Towards a Peer Review of GeoHumanities Projects

1 Leave a comment on block 1 2

 

Follow-up to a workshop at DH2015, Sydney, 29 June 2015

Co-organized by

David Joseph Wrisley, American University of Beirut @DJWrisley
Katherine Hart Weimer, Rice University @kathy_weimer
Karl Grossner, Stanford University @kgeographer

 

BACKGROUND:

The goals, methods, and products of Digital Humanities projects take many forms and may be assessed at various stages of their lifecycle: as planned work described in grant proposals, formally or informally at interim stages of progress, and at completion in independent or publisher-solicited reviews.  Peer review, both formal and informal, has a very meaningful role in scholarship, however, the digital format challenges traditional assessment criteria.  The intent of this document is to provide an opportunity for community input into an expansion of the existing DH Commons review criteria, in particular, for geographically inflected scholarly works. 

The activities and products of DH projects can fall into one or more of several broad categories, including but not limited to: data collections; digital object collections (e.g. images of manuscripts, maps, and other artifacts); scholarly editions of texts; computational analyses of works, events, or activity and visualizations thereof (with or without interactivity); interpretive artistic works; and so on. Interactive products of DH projects may include interpretation and argumentation in extended prose elements, or be an  accompaniment to a traditional print publication.

2 Leave a comment on block 2 1

Clearly there is no set of detailed metrics that can apply to all DH projects and all circumstances of review, but DHCommons, an initiative of CenterNet, has developed a set of review guidelines as a “baseline for commentary” by reviewers, as well as submission guidelines for project teams.

3 Leave a comment on block 3 1

We have taken those guidelines as a starting point, and encourage readers to become familiar with the base DH Commons document noted above.  We have elaborated on some elements with respect to geographically inflected work, and ask you to evaluate their usefulness as criteria in assessing a few varied projects. Our goals are to explore how work in the GeoHumanities require us to expand upon, and perhaps revise if needed, what DHCommons has put forward.  The original DH Commons text appears in plain text below in order to provide context. Note: GeoHumanities-specific elaborations appear in orange, underlined and in bold in this text.

5 Leave a comment on block 5 5

CONTRIBUTION

This section asks reviewers to consider the overall field contribution of the project, paying special attention to gains made following the initial funding period (if applicable). We are particularly interested in how digital methods and modes of presentation offer new ways to address ongoing scholarly conversations or contribute to Digital Humanities methodology.

6 Leave a comment on block 6 0
Questions include

How does the project advance contemporary discussions within its particular subject area?

7 Leave a comment on block 7 1

Does the project fully engage with current scholarship in the field?

8 Leave a comment on block 8 1

 

Apart from work in the subject field, is literature relevant to geospatial methodologies and relevant applications cited? are methods or their application novel?

9 Leave a comment on block 9 0

 

Do the digital methods employed offer unique insights into the project’s key questions?

10 Leave a comment on block 10 4

 

How does mapping and/or geospatial analyses contribute to this work? Is it essential? Is it appropriate for the kind of project?

11 Leave a comment on block 11 1

 

Which geo-humanistic aspect(s) of this project makes the greatest contribution (analysis, narrative, data, etc)?

12 Leave a comment on block 12 1

 

PRESENTATION

Rather than narrowly about web or interface design, this section also asks reviewers to confront the project’s methodological and scholarly aims from a more disciplinary perspective. As much as possible, we want to move from thinking in terms of a form/content divide toward a full consideration of the digital project as purpose-built scholarship. In other words, the digital in digital humanities scholarship should make clear and critical interventions into important field topics, rather than simply being a mode of delivery.

13 Leave a comment on block 13 1
Questions include:

Does the interface effectively communicate and facilitate the goals, purpose, and argument of the project?

14 Leave a comment on block 14 0

 

How do the design and content elements of the project interact and integrate with one another?

15 Leave a comment on block 15 0

 

Discuss usability of the interface(s) from the perspective of a reader/researcher; if possible, also discuss usability from the perspective of current user experience best practices?

16 Leave a comment on block 16 3

 

Are chosen analytic methods (if any) suitable for addressing the research question?

17 Leave a comment on block 17 0

 

Are they used well?

18 Leave a comment on block 18 2

 

Are the data used accurate, open/transparent and appropriate to the stated research questions?

19 Leave a comment on block 19 0

 

Are the maps complete and coherent; do they convey the stated or computed results well?

20 Leave a comment on block 20 0

 

Are they subject to misinterpretation? Are legends clear?

21 Leave a comment on block 21 0

 

Is the uncertainty inherent in all analysis conveyed in the map and/or written results?

22 Leave a comment on block 22 1

 

Do they adhere to basic cartographic principles for legibility and clarity of communication, e.g. “separate meaningful characteristics and […] portray likenesses, differences, and interrelationships” well? [1]

 

23 Leave a comment on block 23 0

PRESERVATION

Preservation can mean a number of different practices in a digital context. From long-term data storage to continued frontend browser optimization, the one thing that unifies preservation as a category is human commitment to upkeep and best practices. Preservation may be addressed in different ways depending on the short or long term goals of the project.

24 Leave a comment on block 24 0
Questions include:

Have relevant best practices and standards been followed for markup and metadata? Has the project used responsive design?

25 Leave a comment on block 25 0

Is documentation available about the project? Is information provided about who, why, when, and how different responsibilities were assigned?

26 Leave a comment on block 26 1

How is the project hosted? Through a university server? A commercial host? A non-profit organization? Is there evidence of ongoing commitment to support of the project at the level of hosting? Is there similar evidence of ongoing support from project personnel?

27 Leave a comment on block 27 2

Is there a preservation and maintenance plan for the interface, software, and associated databases (multiple copies, mirror sites, collaboration with data archives, etc.)? Is the project fully exportable/transferable? Are the data downloadable?

28 Leave a comment on block 28 2

Is the data provenance described and discussed? Is the scope of the data articulated? Are the authors’ manipulations of the data documented in prose or code (and thus reproducible)?  Are reused data appropriately cited?

29 Leave a comment on block 29 1

Is the software being used proprietary, open-source, or editable by multiple programs? Are there clear plans for future accessibility? Will researchers have access to project material and/or metadata outside of a web-based interface? Is any rationale for keeping material/code private provided?

30 Leave a comment on block 30 0

Is there a sustainability plan, if appropriate?

31 Leave a comment on block 31 0

Will the project continue to be updated? If not, is the end of the project discussed?  Is it complete or archivable now?

32 Leave a comment on block 32 0

 

End of document

Source: http://geohum.djwrisley.com/