Collection Development Policy
To be eligible for inclusion in TeachingWithData.org (TwD), all materials must meet the following requirements:
- They are appropriate to fulfilling TwD's Mission.
- They are within TwD's Scope.
- They adhere sufficiently to the TwD Quality Guidelines that are appropriate.
- There is reasonable assurance that they will be maintained.
TeachingWithData.org (TwD), a National Science Foundation-funded, National Science Digital Library Web Pathway, is dedicated to promoting quantitative literacy by helping instructors find high-quality teaching materials that include data for student exercises and research. It will provide content for social science courses at all levels, beginning in high school, and including college-level introductory courses of social science majors and non-majors. Students will be exposed to a broad array of data from censuses and social surveys (many of them funded by NSF), which are not usually available to undergraduate students.
The long-range goals of TwD are to improve quantitative literacy skills in social science students; expose students to the creativity and excitement of empirical research; and assist instructors in development of content by providing easier access to data, tools for analysis and visualization, and pedagogical resources. Creating bridges between social science research and education is fundamental to the TwD mission.
TwD supports its mission by:
- providing access to data, data-related teaching resources, and assessment tools
- providing access to collections of high-quality data and instructional materials, open-access repositories of research articles and reports, and other reliable social science-focused resources
- promoting tools that optimize use of online resources and stimulate educational innovation
- providing a distribution system that facilitate the sharing of resources and tools.
Subject matter. The resources to which TwD provides organized access are referred to as the TwD Collection. This Collection supports the study and teaching of social science concepts through the use of data in the classroom. TwD is especially relevant for courses in demography, economics, history, political science, public policy, social psychology, and sociology, but would also enhance courses in areas such as education, environmental science, family studies, and public health.
Geographic focus and languages. TwD strives to support social science education in the United States in particular. It includes material from around the world, and many of the collections and resources in the Pathway will be of value to educators and instructors outside the United States.
Nature of the materials. The resources that comprise the TwD include:
- instructional materials, such as classroom activities, demonstrations, simulations, interactives, exercises, assignments, data-driven learning modules, tutorials, syllabi, reading lists, curricula, lectures/presentations, instructor guides and manuals, lesson plans, exercises with data, other classroom-ready resources, and tools for analysis and visualization
- pedagogical resources, including teaching techniques, online professional development courses, and educational research
- assessment materials, such as exams, quizzes, questionnaires, self-assessments, answer keys, rubrics, and portfolios
- datasets, electronic numeric data ready for use with statistical packages, and databases
- analysis and visualization tools including charts, tables, graphs, and maps
Access. TeachingWithData.org (TwD) is committed to providing wide access to social science materials. It favors resources and collections that comply with Section 508 of the Rehabilitation Act, which requires that all individuals with disabilities have access to and use of information and data comparable to that provided to individuals without disabilities, unless the provision of such access would impose an undue burden on the provider. The TwD Web site itself is also designed to meet the requirements of the Act.
TwD favors resources that are readily accessible online. Likewise, in selecting Web sites for inclusion in TwD, it favors those that offer digital resources accessible to users everywhere, but may consider sites that require paid or unpaid memberships.
In order to ensure a more complete representation of trustworthy and valuable social science resources, TwD actively encourages instructors, professional organizations, and social science data archives to participate in the Pathway. Some users of TwD, through their affiliation with subscribing institutions or organizations, will have full access to professional organization or data archive materials requiring subscription. Many educational institutions generate data usable for teaching concepts and many belong to data archives such as NORC, ICPSR, and others. These organizations publish and maintain data that may be usable in the creation of data-related resources for teaching quantitative skills. Access to metadata records that provide information about data archive materials are available to all users of TwD.
Sources of content. The sources of TwD content include:
- social science-focused collections developed by professional organizations, universities, and grant-funded projects
- Web sites created by reputable and reliable organizations or individuals
- publishers of social science content
- social science-focused companies such as educational software developers
- social science data archives
- Government agencies and organizations
- other digital libraries.
Materials considered unsuitable for the TeachingWithData.org (TwD) Collection. The TwD Collection will exclude the following:
- data whose source is unknown or there is evidence of unscientific practice in collection and/or analysis (See examples of what would be considered unscientific practice.)
- data that are produced by an advocacy group and therefore may be seen as biased
- unsecured data that could reveal the identity of a study's respondents
- socially undesirable materials, e.g., pornography
- Web sites created primarily to advertise a service or product.
TeachingWithData.org (TwD) has resource quality guidelines for Collection materials in order to assist in identification and selection, define a level of expectation/performance, and provide guidance in development. These quality guidelines apply to Collection materials as defined in the TwD Mission and Scope.
Because TwD includes many types of materials intended to address the needs of a wide range of audiences, it is not expected that every resource in the Collection will meet all of the TwD quality guidelines at once. In particular, the level of review of individual resources will necessarily vary. In considering collections of materials intended for use in high school and undergraduate educational settings, in which students generally work within well-defined curricula directed by an instructor, we strongly favor collections that have an effective review process for selecting resources of high quality and pedagogical effectiveness. The expectation is that Collection materials, in keeping with their specific natures, will adhere as much as possible to the quality characteristics described below.
The resource is relevant to social science education.
The TwD statements of Mission and Scope define parameters of social science education within which it strives to offer materials. Collection materials should fall within this mission and scope. TwD collects materials designed for teaching with data and learning relevant to social science education. Background material, reference works, and other supporting resource types are not excluded if they are within scope.
The resource is accurate.
TwD materials should adhere to sound pedagogical principles and practices. Collection materials should meet as many of the following criteria as possible:
- social science concepts are taught using data
- the resource identifies methodology, social science concept, and hypothesis (as appropriate)
- links to related Web sites are provided.
The origin of the resource is attributed.
Resource attribution may be to a person or persons, to an organization, or to both. The rationale for this guideline is that knowledge about the origin of Collection materials (authors, sponsors, or publishers) helps TwD Collection developers and users judge the reliability of material. This means the following, as appropriate, is known about the material:
- name of the resource creator (individual and/or sponsoring institution)
- contact information for the resource creator or maintainer
- date of resource creation or most recent update.
The resource is robust and functional.
Collection materials should be fully operational when users access them. Therefore, Collection materials should:
- be free of conspicuous bugs, defects, and nonworking elements (links that don't work, graphics that don't display, applets that don't run) that inhibit their intended use
- work on current versions of major Web browsers, under operating systems used widely in educational settings
- allow printing of text and graphic elements.
The resource is educationally documented.
A well-documented instructional resource includes:
- educational level for which the resource is recommended
- skills and knowledge needed prior to working with the resource
- learning objectives
- commonly observed mistakes or misconceptions
- notes on instructional strategies
- alignment with educational standards, where appropriate.
The resource is pedagogically effective.
Resources that are pedagogically effective tend to have the following characteristics:
- The instructional time required for use of the resource is commensurate with the importance and magnitude of the learning achieved.
- The level of difficulty is appropriate for the target audience.
- The instructional strategies build toward mastering the stated learning objectives.
- Student learning achieved by the use of the resource is assessable; ideally, the resource includes assessment options with or within the material and guidance for teachers, such as an answer key or scoring rubric.
The resource makes an important or significant contribution to learning.
Collection materials should deal with content, skills, or knowledge that are likely to be important to a significant fraction of social science educators and students. While the material itself may not state why the learning made possible by the resource is important or significant, one should be able to make a case for it--on grounds, for example, that the material supports:
- learning required by national or state educational standards
- learning on a topic of societal importance
- mastering of foundational skills needed for advancement in a discipline or career
- understanding of a concept central to a social science discipline
- understanding of linkages and interactions among or within social science disciplines, or between social science and other disciplines.
The documentation provided for the resource is complete.
In addition to the above-mentioned educational documentation and attribution as to origin, there are other kinds of documentation that particular types of Collection materials should provide.
Documentation of resources with specific technical requirements should include:
- specification of hardware required to use the material
- specification of software, including plug-ins, required to use the material, including links or contact information for users needing to acquire such software
- disclosure of any known hardware or software incompatibilities
- for software or complex resources, a clear and comprehensive user's guide
- file sizes of downloadable materials or large viewable files.
Documentation of data should include:
- basic information (or a link to information) about the instrument that collected the data, including how the instrument works
- for individual datasets, information about when, where, and by whom the data were collected
- for derived data products or merged datasets, links to the data archive or other source of description about how the data were processed or merged, and information about when and where the data were collected
- links to sources of original data and tools to access the data
- the codebook or link to it
- geocoding of the data, where possible.
Educators and learners will find the resource easy to use.
Since an important component of the TeachingWithData.org (TwD) mission is provision of organized access to high-quality materials that support innovations in teaching and learning at all levels, instructional materials should be easy to use for educators and learners. Easy-to-use resources tend to include the following characteristics:
- Navigation pathways are self-evident and intuitive
- Preparation time and effort required by an instructor is not excessive relative to the educational potential of the material.
- Methods are provided for adapting to students' and teachers' individual differences, interests, abilities, and needs. Ideally, the resource will provide any or all of the following:
- teacher-editable versions of student handouts (MS Word or PDF documents)
- several levels of difficulty or complexity in student activities
- links to maps or local datasets for use in different locations.
- Information about ease of use is provided, if possible. There is particular value in feedback:
- from educators who have observed learners interacting with the material
- from learners who have used the material
- from an expert in usability of the material.
The resource is engaging for learners.
TwD favors instructional materials that engage and motivate learners to:
- show curiosity about a topic or the material
- ask questions or seek out further information
- perform at a high level (i.e., do their personal best work).
Ideally, instructional materials should provide feedback:
- from educators who have observed learners interacting with the material
- from learners who have used the material
- from educational specialists familiar with the target audience.
The resource is free of distracting or off-topic advertising.
Collection materials should be as free of distracting advertising as possible, especially advertising that is irrelevant or that interrupts or interferes with the use of the material.
The resource is easily accessible to users.
To ensure wide accessibility and use, TeachingWithData.org (TwD) favors materials that are free or available at low cost. TwD does not, however, exclude materials potentially useful to specific audiences on the grounds that fees, subscriptions, or other access requirements have been put in place by the owner. We do require that such access requirements be made known to TwD at the time of accessioning.
Portions of this statement are adapted from other NSDL Pathways, including DLESE.