Open assessment method: Difference between revisions

From Opasnet
Jump to navigation Jump to search
(contents from Help:open risk assessment and Help:Open assessment quiz moved here)
 
(CII comments added)
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''Open risk assessment''' (ORA) is a description of a certain risk. It has been produced in the internet using a process where stakeholders and other people have been able to participate. They have been given an opportunity to comment and edit its contents already from an early phase of the process.
{{method}}
[[Category:Open assessment]]
This page is about the specifics of the '''open assessment method'''. To read a general description, see [[open assessment]].


==Structure of an ORA==


===Variables===
:'''Open assessment''' (previously also known as '''pyrkilo''') is a method that attempts to answer the following research question and to apply the answer in practical assessments:
::'''How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?'''
:[[Open assessment]] can also refer to the actual making of such an assessment (precisely: open assessment process), or the end product of the process (precisely: open assessment product or report). Usually, the use of the term open assessment is clear, but if there is a danger of confusion, the precise term (open assessment method, process, or product) should be used. In practice, the assessment processes are performed using Internet tools (notably [[Opasnet structure|Opasnet]]) among traditional tools. Stakeholders and other interested people are able to participate, comment, and edit its contents already since an early phase of the process. Open assessment is based on a clear information structure and [[:en:scientific method|scientific method]] as the ultimate rule for [[dealing with disputes]]. {{reslink|Rename ORA into OA}}


'''Variables''' are the basic building blocks of an ORA. They describe some real-world things such as emissions of a pollutant, number of health effects due to a certain exposure in a population, or costs of a policy intervention. Variables are connected to each other with links that indicate a causal connection. The variables and the links together form a causal diagram that is the skeleton of an ORA.
==Question==


Some variables have a special meaning or purpose. ''Indicators'' are outcome variables that are the main interest of the ORA, i.e. their values are the actual result of the ORA. ''Policy options'' are variables that describe a group of possible actions such as installing (or not installing) a filter to the end of a smoke stack. All other variables are basically needed to estimate, how the policy options considered will affect the indicators. This information is then used in the subsequent decision-making.
How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?


===Attributes===
==Answer==


{{Template:Attributes of a variable}}
* There must be a clear information structure.
** To operationalise the information structure, there must individual objects that each belong to a group of objects defined by a [[universal object]]. For discussion about what these objects should be, see [[Universal object]].
* All objects must be subject to the [[:en:scientific method|scientific method]]. The scientific method is a method of discovering [[wiktionary:knowledge|knowledge]] about the natural world based in making [[wiktionary:falsifiable|falsifiable]] predictions ([[:wiktionary:hypothesis|hypotheses]]), testing them [[wiktionary:empirically|empirically]], and developing peer-reviewed theories that best explain the known data ([[wiktionary:scientific method|Wiktionary]]).
** To operationalise the scientific method, [[:en:Pragma-dialectics|pragma-dialectical argumentation theory]] is applied for discussions.
* There must be a website, [Opasnet structure|Opasnet]], which contains the functionalities needed by the method and the contents produced by the method.


===Categories===
==Rationale==


'''Categories''' are used to manage the variables. Each variable can belong to one or more categories. Each category can also belong to  one or more categories that are more general the itself. As an example, a variable ''Fine particle concentration in Helsinki'' belongs to a category ''Fine particles'', which belongs to a category ''Air pollutants''. Categories are used in the same way as in Wikipedia [[en:Help:Categories]]. However, there some specific categories that should be used systematically to categorise variables ORAs. Each variable shold belong to one of these categories, or their subcategories. The main categories are described below.
* Tapscott and Williams provide insight into why mass collaboration work is a better way of producing information products than traditional systems.
*[[:Category:Activities | Activities]] describe any human activity that result in releases of  pollutants or other hazards into the environment.
* James Surowiecki describes how a group of people are in some conditions better in predicting things than an expert.
*[[:Category:Releases | Releases]] describe releases of  pollutants or other hazards into the environment.
* Jimmy Wales has demonstrated the power of mass collaboration in his [[:en:Main Page|Wikipedia]].
*[[:Category:Pollutants | Pollutants]] describe what the pollutants or other hazards are. All variables that relate to a particular pollutant should be indexed with the category of that pollutant, and that category should be indexed to Pollutants.
*[[:Category:Concentrations in the environment | Concentrations in the environment]] describe
*[[:Category:Exposures | Exposures]]
*[[:Category:Health effects | Health effects]] exposure-reponse functions and health effects in humans.
*[[:Category:Costs and valuations|Costs and valuations]] describes value-laden summaries of health and other effects.


== Open assessment quiz ==


'''Open assessment quiz''' is a way to find out, how much you actually know about the open assessment or the pyrkilo method. Some basic knowledge is needed to be able to effectively edit pages. Everyone can still contribute by providing data or information, or commenting any existing content. However, if you do not know anything about the method that is applied in these pages, we recommend that you wrap your contributions inside a comment (the button with a thin blue line, above the edit window). You can also start by using the [[contribution tool]] as soon as we get it running.
A website is needed for collaboration about the contents of assessments. Collaboration may happen in one of many languages. Assessments contain numerical estimates and computation, and the results must be stored into a place where they can be easily accessed and downloaded for further use. In addition, a file management system is needed for data, model, and other files.


Select the correct answers for the questions below and send them via email to jouni.tuomisto(at)ktl.fi. You will get your score and right answers as reply.
* There must be a clear and universal information structure.  
* The method must be based on individual objects that have a research question. The essence of each object is to try and find such an answer to the question that holds against scientific criticism.


True or false?
'''Critical questions (asked by CII students in 2011):
# Open assessment is called open because the results are openly available.
 
# All open assessments consist of variables.
* Who decides about content?
# The result of a variable is always numerical.
* Is it peer reviewed?
# The attributes of a variable are name, scope, data, and result.
* How do you know about the expertise of a contributor?
# Open assessment is the same thing as a risk assessment.
* How can you consider all contributions acceptable a priori?
# Cause-effect relations form the basic structure of an open assessment.
* How can you put drafts out?
# Assessments and variables are product objects.
* People will learn erroneous information.  
# Classes are process objects.
Answers: it is all about learning, not about having the perfect information available in the beginning. Also it is a process that improves. Find an example of a page that has poor scientific quality but is useful.
# Classes are particular kinds of sets, in which all the items share some explicitly defined properties.
 
# NOAEL (no observed adverse effect level) of a particular chemical can be defined as a variable.
==See also==
# NOAEL has a causal relation to the reference dose (RfD) of the chemical.
 
# NOAEL has a causal relation to the health effects of the chemical.
* [[Open Assessors' Network]]
# All you need to know to calculate the result of a variable is the information described in the definition attribute of the same variable.
* [[Opasnet structure]]
# The result cannot be a single number, it is always a distribution.
 
# Health impact assessment and risk assessment are process objects.
==References==
# A process object does have a result attribute, and the result is always a universal product object.
 
# Any particular piece of reality is always described as a variable.
<references/>
# The clairvoyant test examines whether a scope is clear enough.
# If several variables contain the definition of PM<sub>2.5</sub>, you can create a class from the definition.
# When starting an open assessment, the first thing is to define the purpose of the assessment.
# Anyone can participate in a properly open assessment.
# A variable cannot connect to the data attribute of another variable, but only to the causality attribute.
# Causality attribute only contains the list of parent variables. It does not contain data about the causalities; this data belongs to the data attribute.
# References is also an attribute.
# The scope of a variable in your assessment is called "persistent organic pollutant concentrations in adults in Finland", and someone adds data about DDT into the variable. You are allowed to prevent that because its your assessment.
# You are allowed to prevent that because the variable is about dioxin, not DDT.
# You are not allowed to prevent that, but you can start a discussion against the addition.
# You are likely to win, because you can use the practical argument that such a change into this critical variable would make it incoherent in your assessment, and several other assessments as well.
# You must use your real name when making contributions to open assessments.
# When an assessment is finished, the assessment and its variables are fixed and no further edits are allowed.

Latest revision as of 20:13, 14 August 2012

This page is about the specifics of the open assessment method. To read a general description, see open assessment.


Open assessment (previously also known as pyrkilo) is a method that attempts to answer the following research question and to apply the answer in practical assessments:
How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?
Open assessment can also refer to the actual making of such an assessment (precisely: open assessment process), or the end product of the process (precisely: open assessment product or report). Usually, the use of the term open assessment is clear, but if there is a danger of confusion, the precise term (open assessment method, process, or product) should be used. In practice, the assessment processes are performed using Internet tools (notably Opasnet) among traditional tools. Stakeholders and other interested people are able to participate, comment, and edit its contents already since an early phase of the process. Open assessment is based on a clear information structure and scientific method as the ultimate rule for dealing with disputes. R↻

Question

How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?

Answer

  • There must be a clear information structure.
    • To operationalise the information structure, there must individual objects that each belong to a group of objects defined by a universal object. For discussion about what these objects should be, see Universal object.
  • All objects must be subject to the scientific method. The scientific method is a method of discovering knowledge about the natural world based in making falsifiable predictions (hypotheses), testing them empirically, and developing peer-reviewed theories that best explain the known data (Wiktionary).
  • There must be a website, [Opasnet structure|Opasnet]], which contains the functionalities needed by the method and the contents produced by the method.

Rationale

  • Tapscott and Williams provide insight into why mass collaboration work is a better way of producing information products than traditional systems.
  • James Surowiecki describes how a group of people are in some conditions better in predicting things than an expert.
  • Jimmy Wales has demonstrated the power of mass collaboration in his Wikipedia.


A website is needed for collaboration about the contents of assessments. Collaboration may happen in one of many languages. Assessments contain numerical estimates and computation, and the results must be stored into a place where they can be easily accessed and downloaded for further use. In addition, a file management system is needed for data, model, and other files.

  • There must be a clear and universal information structure.
  • The method must be based on individual objects that have a research question. The essence of each object is to try and find such an answer to the question that holds against scientific criticism.

Critical questions (asked by CII students in 2011):

  • Who decides about content?
  • Is it peer reviewed?
  • How do you know about the expertise of a contributor?
  • How can you consider all contributions acceptable a priori?
  • How can you put drafts out?
  • People will learn erroneous information.

Answers: it is all about learning, not about having the perfect information available in the beginning. Also it is a process that improves. Find an example of a page that has poor scientific quality but is useful.

See also

References

Retrieved from "https://en.opasnet.org/index.php?title=Open_assessment_method&oldid=25916"