User Guide¶
Introduction¶
The Panoptes Client provides high level access to the HTTP API for common project management tasks. The client module provides a set of classes which act as an object-relational mapping (ORM) layer on top of the API, allowing you to perform tasks such as creating/uploading subjects, retiring subjects, and downloading data exports without having to concern yourself with the low level detail of how the API functions.
Most of the classes you’ll need can be imported directly from the
panoptes_client
package; see the module reference
for a complete list.
Of special note is the Panoptes
class which provides the
Panoptes.connect()
method. This method must be called to log into the
API before you can perform any privileged, project owner-specific actions
(though some things, such as listing public projects or available workflows,
can be done anonymously, without logging in).
Most of the classes you’ll be using are subclasses of the abstract
PanoptesObject
class. Any references in this documentation to
“Panoptes object classes” or “model classes” refer to these subclasses. You
should familiarise yourself with the methods that Panoptes object classes all
have, in particular PanoptesObject.save()
and
PanoptesObject.where()
.
You might also want to refer to the Panoptes API documentation as this lists the full options and allowed values for many of the methods in this module – many method arguments are simply passed to the API as-is, with the API performing server-side validation. The API documentation also lists the full attributes for each class; these are not included in this documentation.
Installation¶
Install latest stable release:
$ pip install panoptes-client
Or for development or testing, you can install the development version directly from GitHub:
$ pip install -U git+https://github.com/zooniverse/panoptes-python-client.git
Upgrade an existing installation:
$ pip install -U panoptes-client
The Panoptes Client is supported on all versions of Python 2 and 3, from Python 2.7 onwards.
Uploading non-image media types¶
If you wish to upload subjects with non-image media (e.g. audio or video),
you will need to make sure you have the libmagic
library installed. If you
don’t already have libmagic
, please see the dependency information for
python-magic for more
details.
Usage Examples¶
Tutorial: Creating a new project¶
Once you have the client installed, you can import the modules you need from the
panoptes_client
package. For this tutorial, we’re going to log into the
Panoptes API, create a project, create a subject set, and finally create some
subjects to go in the new set. Let’s start by importing all the classes we’ll
need for that:
from panoptes_client import Panoptes, Project, SubjectSet, Subject
Now that we’ve imported all that, we can use the Panoptes.connect()
method to log in:
Panoptes.connect(username='example', password='example')
Next we will create our new project. All we need to do is instantiate a new
instance of Project
, set some required attributes, and then save
it, like so:
tutorial_project = Project()
tutorial_project.display_name = 'Tutorial Project'
tutorial_project.description = 'My first project created in Python'
tutorial_project.primary_language = 'en'
tutorial_project.private = True
tutorial_project.save()
Now if you log into the Zooniverse project builder you should see the new project listed there. Next we will create a subject set in the same way:
subject_set = SubjectSet()
subject_set.links.project = tutorial_project
subject_set.display_name = 'Tutorial subject set'
subject_set.save()
Here you’ll notice we set the subject_set.links.project
attribute. links
is a special attribute that handles connecting related Panoptes objects to each
other. You can directly assign a Panoptes object instance, as above, or you can
assign an object’s ID number if you have it. As well as assigning objects, you
can use links
to access related objects. Now that we’ve created our new
subject set, we will also see a link to it on tutorial_project
if we reload
it:
tutorial_project.reload()
print(tutorial_project.links.subject_sets)
This would output something like this:
[<SubjectSet 1234>]
Showing a list of the linked subject sets (containing only our new set in this
case). Here 1234
is the internal ID number of the subject set (also
accessible as subject_set.id
), so the exact result you get will be slightly
different.
Now that we have a subject set, let’s create some subjects and add them to it.
For this tutorial, we’ll assume you have a dict
containing filenames
and subject metadata. In reality you might load this from a CSV file, or query a
database, or generate it in any number of different ways, but this would be
outside the scope of this tutorial:
subject_metadata = {
'/Users/me/file1.png': {
'subject_reference': 1,
'date': '2017-01-01',
},
'/Users/me/file2.png': {
'subject_reference': 2,
'date': '2017-01-02',
},
'/Users/me/file3.png': {
'subject_reference': 3,
'date': '2017-01-03',
},
}
Now we create a Subject
instance for each one:
new_subjects = []
for filename, metadata in subject_metadata.items():
subject = Subject()
subject.links.project = tutorial_project
subject.add_location(filename)
subject.metadata.update(metadata)
subject.save()
new_subjects.append(subject)
Saving the subject will create the subject in Panoptes and then upload the
image file. The Subject.add_location()
method prepares files to be
uploaded. You can give it a string, as above, to point to a path on the local
filesystem, or you can give it an open file
object, or a
dict
for remote URLs. See the Subject.add_location()
documentation for examples.
Note that by default the metadata
attribute is an empty dict
,
so in this example we just call dict.update()
to merge it with our
existing metadata. You can also set individual keys as normal:
subject.metadata['my_metadata'] = 'abcd'
Or you can leave it empty if you don’t need to set anything.
All that’s left to do now is to link our new subjects to our new subject set.
That can be done with the SubjectSet.add()
method:
subject_set.add(new_subjects)
That takes the list of subjects and links them all in one go. This is the preferred way of doing it if you have several subjects to link (because it’s faster than making several separate calls), but you can also link subjects one at a time if you need to:
subject_set.add(subject1)
subject_set.add(subject2)
And that’s all there is to it! Your new subjects are now linked to the new subject set.
Tutorial: Adding a Workflow to Caesar¶
For this tutorial, we will connect to Caesar and add workflow to Caesar in 2 ways (via Caesar or via Workflow). We start by importing all the classes we’ll need:
from panoptes_client import Panoptes, Workflow, Caesar
Now that we’ve imported all that, we can use the Panoptes.connect()
method to log in (see above tutorial).
Next we can instantiate an instance of :py:class`.Caesar`:
caesar = Caesar()
Note that the token from coming from Panoptes.connect()
will also get us connected to Caesar.
We can add workflow to Caesar using this instace of :py:class`.Caesar`, assuming you have a workflow_id handy:
caesar.save_workflow(1234)
Another way we can do this is via :py:class`.Workflow`. We can do this by first instantiating an instance of :py:class`.Workflow` with provided workflow_id:
workflow = Workflow(1234)
We can then add this workflow to Caesar:
workflow.save_to_caesar()
Tutorial: Retiring and Unretiring Subjects¶
For this tutorial, we’re going to retire and unretire subjects in a given workflow. We start by importing all the classes we’ll need:
from panoptes_client import Panoptes, Workflow, Subject, SubjectSet
Now that we’ve imported all that, we can use the Panoptes.connect()
method to log in (see above tutorial)
Next we can instantiate an instance of :py:class`.Workflow`, assuming you have a workflow_id handy:
workflow = Workflow('1234')
We can retire subjects by doing any one of the following, for these examples, we have a Subject with id 4321:
workflow.retire_subjects(4321)
workflow.retire_subjects([4321])
workflow.retire_subjects(Subject(4321))
workflow.retire_subjects([Subject(4321)])
Similarly, we allow the ability to unretire subjects by subject by doing any one of the following, for these examples, we use a Subject with id 4321:
workflow.unretire_subjects(4321)
workflow.unretire_subjects([4321])
workflow.unretire_subjects(Subject(4321))
workflow.unretire_subjects([Subject(4321)])
We also allow the ability to unretire subjects by SubjectSet by doing any on of the following, for these examples, we use a SubjectSet with id 5678:
workflow.unretire_subjects_by_subject_set(5678)
workflow.unretire_subjects_by_subject_set([5678])
workflow.unretire_subjects_by_subject_set(SubjectSet(5678))
workflow.unretire_subjects_by_subject_set([SubjectSet(5678)])
Other examples¶
Print all project titles:
for project in Project.where():
print(project.title)
Find a project by slug and print all its workflow names:
project = Project.find(slug='zooniverse/example')
for workflow in project.links.workflows:
print(workflow.display_name)
List the subjects in a subject_set:
subject_set = SubjectSet.find(1234)
for subject in subject_set.subjects:
print(subject.id)
Add subject set to first workflow in project:
workflow = project.links.workflows[0]
workflow.links.subject_sets.add(subject_set)
Project owners with client credentials can update their users’ project settings (workflow_id only):
Panoptes.connect(client_id="example", client_secret="example")
user = User.find("1234")
project = Project.find("1234")
new_settings = {"workflow_id": "1234"}
ProjectPreferences.save_settings(
project=project,
user=user,
settings=new_settings,
)
Alternatively, the project ID and user ID can be passed in directly if they are already known:
ProjectPreferences.save_settings(
project=project_id,
user=user_id,
settings=new_settings,
)
Importing iNaturalist observations to Panoptes as subjects is possible via an API endpoint. Project owners and collaborators can use this client to send a request to begin that import process:
# The ID of the iNat taxon to be imported
taxon_id = 1234
# The subject set to which new subjects will be added
subject_set_id = 5678
Inaturalist.inat_import(taxon_id, subject_set_id)
As an optional parameter, the updated_since timestamp string can be included and will filter obeservations by that parameter:
Inaturalist.inat_import(taxon_id, subject_set_id, '2022-10-31')
Be aware that this command only initiates a background job on the Zooniverse to import Observations. The request will return a 200 upon success, but there is no progress to observe. You can refresh the subject set in the project builder to see how far along it is, and the authenticated user will receive an email when this job is completed.
Other examples Caesar features by Workflow¶
Most Caesar use cases are usually through a workflow: the following are examples of Caesar functions that can be done via Workflow.
Add Caesar Extractor by Workflow:
workflow = Workflow(1234)
workflow.add_extractor('question', 'complete', 'T1', {'if_missing' : 'ignore'})
Add Reducer by Workflow:
external_reducer_attributes = {
'url': 'https://aggregation-caesar.zooniverse.org/reducers/optics_line_text_reducer',
'filters': {
'extractor_keys': ['alice']
}
}
workflow.add_caesar_reducer('external', 'alice', external_reducer_attributes)
Adding Subject Rules by Workflow. When creating a rule, the condition_string argumentis a stringified array with the first item being a string identifying the operator. See https://zooniverse.github.io/caesar/#rules for examples of condition strings:
condition_string = '["gte", ["lookup", "complete.0", 0], ["const", 30]]'
workflow.add_caesar_rule(condition_string, 'subject')
Adding Subject Effect for a Subject Rule with id 1234 by Workflow. Ths particular effect being created will retire subjects early due to a consensus.
workflow.add_caesar_rule_effect('subject', 1234, 'retire_subject', {'reason' : 'consensus'})