PubChemRDF 1.7β has been released

A significant update has been made to PubChemRDF, machine-readable PubChem data formatted using the Resource Description Framework (RDF) (https://www.w3.org/RDF/).  (If you have never heard about PubChemRDF before, please read this PubChem blog first.)

What is PubChemRDF?

RDF is a World Wide Web Consortium (W3C) standard model for data interchange on the web.  In RDF, knowledge is expressed as statements, each of which consists of three discrete parts: a subject, an object, and a predicate that specifies the relationship between them.  So, the trio of these parts is called a triple.  For example, the sentence “asbestos can cause mesothelioma” consists of “asbestos” (subject), “mesothelioma” (object) and “can cause” (predicate).  Similarly, the sentence “ethanol is metabolized to acetaldehyde” can be broken down into a triple of “ethanol” (subject), “acetaldehyde”, “is metabolized to” (predicate).  In essence, RDF expresses knowledge into a directed, labeled graph.

PubChemRDF refers to the RDF-formatted PubChem data.  It contains information on various entities in PubChem (chemicals, bioassays, genes, proteins, pathways, literature, etc.) and their relationships.  With PubChemRDF, researchers can work with PubChem data using Semantic Web technologies (https://en.wikipedia.org/wiki/Semantic_Web).  In addition, PubChemRDF facilitates PubChem data sharing, analysis, and integration with data from other resources.

PubChemRDF 1.7-beta

What’s new in PubChemRDF 1.7β?

  • Updated vocabularies
    To define the semantic relationships (that is, predicates) between entities (subjects and objects), PubChemRDF uses pre-existing, domain-specific ontological frameworks (rather than creating new ones), such as Chemical Entities of Biological Interest (ChEBI) , CHEMical INFormation ontology (CHEMINF), Protein Ontology (PRO), Gene Ontology (GO), BioAssay Ontology (BAO), among others.  Since PubChemRDF was first introduced, some terms in these ontologies were deprecated or replaced with new ones.  These changes are now reflected in PubChemRDF 1.7β.
  • New subdomain
    In PubChemRDF 1.7β, a new subdomain, called Pathway, is added to encode information on biological pathways and their relationship with genes, proteins, and chemicals. This Pathway subdomain supersedes the BioSystem subdomain used in the previous versions of PubChemRDF.
  • GI to accession
    In the previous versions, numeric identifiers called GI numbers were used to denote proteins or genes.  However, NCBI phased out the use of GI numbers in its databases, as explained in a series of blog posts.  Accordingly, changes have been made to allow one to access PubChemRDF data using the ‘accession’ identifiers.

Where can I learn more about PubChemRDF 1.7β?

To learn more about this topic, please read the following:

Introducing PubChem Pathway Pages

PubChem Pathway Pages are now available. Each PubChem Pathway page provides information about chemicals, proteins, genes, and diseases involved in or associated with the biological pathway, which can be very important to provide a context to observed biological activity. In addition, all pathways associated with a given chemical, protein or gene are summarized on the corresponding page.

PubChem Pathways

All content comes from existing Pathway resources without any attempt to merge or combine them.  Each page for a given pathway can be accessed via an URL of this form:

https://pubchem.ncbi.nlm.nih.gov/pathway/SOURCE:PATHID

where SOURCE is the information source for the pathway and PATHID is the record identifier used by the source.  For example, the following URL directs to the pathway page for the citric acid cycle in human and mouse (ID: SMP0000057 and SMP0063477, respectively) from PathBank:

https://pubchem.ncbi.nlm.nih.gov/pathway/PathBank:SMP0000057  (for human)

https://pubchem.ncbi.nlm.nih.gov/pathway/PathBank:SMP0063477  (for mouse)

PubChem Pathways supersedes the NCBI BioSystems database, which is no longer being updated. If you have a NCBI BioSystems identifier (BSID) and the page exists in PubChem Pathways, you can access the corresponding PubChem Pathway page via, for example:

https://pubchem.ncbi.nlm.nih.gov/pathway/BSID:703092

Chemicals, proteins, and genes presented on PubChem Pathway pages are linked to corresponding PubChem pages, providing quick access to more detailed information on these entities.  In addition, the Pathway page provides information on the interactions or reactions among these entities.  The PubChem Pathway pages are searchable within PubChem Search.

Lastly, PubChem Pathway information is integrated with the NCBI Gene database (https://www.ncbi.nlm.nih.gov/gene/).  For instance, the following web page presents all pathways associated with the human EGFR (NCBI Gene ID: 1956) in PubChem Pathways:

https://www.ncbi.nlm.nih.gov/gene/1956#pathways

To learn more about the PubChem Pathway page, please read this Help page (https://pubchemdocs.ncbi.nlm.nih.gov/pathways).

Molecular property links to SpringerMaterials are now in PubChem

More than 32,000 compounds in PubChem now have links to hundreds of chemical and physical properties pertinent to chemistry, material science, physics, and other related fields available from SpringerMaterials (see this press release).  These links will help you quickly locate articles for the property in question.

Chemicals with SpringerMaterials links will contain a “SpringerMaterials Properties” section in the “Chemical and Physical Properties” table of contents.  This provides a list of chemical properties available at SpringerMaterials for this compound.  For example, the following link shows the list of the material properties for benzene (Figure 1).

https://pubchem.ncbi.nlm.nih.gov/compound/benzene#section=SpringerMaterials-Properties

SpringerMaterials data integration with PubChem
Figure1. The SpringerMaterials properties for benzene (CID 241) (https://pubchem.ncbi.nlm.nih.gov/compound/241#section=SpringerMaterials-Properties). The 13C nuclear magnetic resonance spectrum link takes you to the entry at the SpringerMaterials site.

Clicking on one of the properties in this list directs you to the SpringerMaterials web page showing a list of articles containing detailed information on that property.  Currently, more than 32,000 compounds have links to SpringerMaterials property data.  A list of these compounds is available through the PubChem Sources page or via the PubChem Classification Browser.

The addition of Springer Materials links to PubChem assists users in finding important data and literature available for chemicals.

 

Integration of WIPO’s PATENTSCOPE data with PubChem

The World Intellectual Property Organization (WIPO) is an international organization that aims to promote the protection of intellectual property throughout the world.  WIPO provided PubChem with more than 16 million chemical structures searchable in its patent database called PATENTSCOPE (see this press release).

PubChem-WIPO data integration

For each of the chemical structures contributed by WIPO, PubChem provides a direct link to PATENTSCOPE, which allows users to perform searches for patent documents relevant to that chemical structure.  For example, the following URL directs users to the “WIPO PATENTSCOPE” section of the cholesterol-lowering medication atorvastatin (CID 60823):

https://pubchem.ncbi.nlm.nih.gov/compound/60823#section=WIPO-PATENTSCOPE

By clicking the direct link presented in this section, users can search PATENTSCOPE for patent documents relevant to CID 60823 and further analyze returned hits using the tools available at PATENTSCOPE.

A list of chemical structures contributed by WIPO can be obtained through the PubChem Source page for PATENTSCOPE:

https://pubchem.ncbi.nlm.nih.gov/source/23607

The integration of WIPO’s chemical information with PubChem makes it easier for PubChem users to find pertinent patent information about chemicals.

Webinar on current access to TOXNET resources

NLM staff will participate in the next American Chemical Society webinar for the chemical information and cheminformatics community: An Overview of NLM’s Post-TOXNET Resources. TOXNET (the TOXicology Data NETwork) was retired in December 2019 as part of the reorganization associated with the NLM Strategic Plan. Most of TOXNET’s databases have been incorporated into other NLM resources such as PubChem and Bookshelf, or continue to be available elsewhere. This webinar will show you where to go now for TOXNET information.

  • Date and Time: Tuesday, March 17 at 1:00pm EDT.
  • Register 

A live Q&A session will follow the webinar.

PubChem Periodic Table and Element pages

UPDATE (July 29, 2020): See also the paper published in Chemistry Teacher International (doi:10.1515/cti-2020-0006).

The periodic table of chemical elements is one of the most recognized tools in science.  As we mark the 150th anniversary of the periodic table, the scientific community has declared 2019 to be “The International Year of the Periodic Table”.  PubChem is celebrating by launching the PubChem Periodic Table and corresponding Element pages.

Periodic_Table

While PubChem provides each chemical its own page, you can find elements there too.  Such pages are not suited for displaying information specific to elements (such as electronegativity and electron configuration).  The PubChem Periodic Table and Element pages help you navigate the abundant chemical element data available within PubChem, while providing a convenient entry point to explore additional information, such as bioactivities, health and safety data, available in PubChem Compound pages for specific elements and their isotopes.

PubChem Element Page

PubChem Element page content comes from scientific articles and various authoritative data sources, such as the International Union of Pure and Applied Chemistry (IUPAC), National Institute of Standard and Technology (NIST), International Atomic Energy Agency (IAEA), Jefferson Laboratory, and Los Alamos National Lab.

The PubChem Periodic Table provides three distinct views.  Table View is the traditional periodic table any scientist would instantly recognize.  List View provides a summary view, allowing you to see all properties available for each element at once.   Game View, added as an educational feature, helps test your knowledge of element names and symbols.

Clicking an element in the PubChem Periodic Table directs you to the corresponding Element page.  This page presents a wide variety of element information, including atomic properties (electron affinity, electronegativity, ionization potential, oxidation states, electron configuration, etc.) as well as isotopes, history, uses, and, most importantly, information source.  The element page can also be reached directly via URLs that includes atomic number, symbol, or name (all case insensitive).  For example, the following URLs are for the Element page for carbon:

https://pubchem.ncbi.nlm.nih.gov/element/Carbon

https://pubchem.ncbi.nlm.nih.gov/element/C

https://pubchem.ncbi.nlm.nih.gov/element/6

In addition, the data presented in the Periodic Table and Element pages are also available through programmatic access, using PUG-REST and PUG-View.

PubChem Homepage has a new look and feel!

We’ve redesigned PubChem’s homepage to give you easier access to the information you need, where you need it. The mobile-friendly, responsive design works on the device you want to use. And the streamlined, intuitive interface puts the data you need at your fingertips.

Here are some of the changes you can expect to see at the new PubChem homepage:

New PubChem Homepage

The menus at the top of the page and the sidebar have been replaced with a minimal set of important links. These links include “About,” “Blog,” “Submit,” and “Contact.” The “About” link will bring you to the PubChem Docs site, where you can find an exhaustive list of PubChem services and documentation.

In addition, data count and data source statistics have been highlighted. Each also includes a link you can follow to get more information on these statistics.

Finally, we’ve improved PubChem’s search New PubChem Homepagecapabilities. The three search boxes for compounds, substances and bioassays have been replaced with a single search box that covers all search types. Search results from the formerly separate search types (compound, substance, and bioassay) have also been integrated into a single search results display. In addition, search in PubChem now directly supports formula and structure search. We have many more details to share with you about the new PubChem search in a separate blog post, so keep your eyes open for that!

We want to know what you think!

PubChem’s new look and feel is a big step forward for PubChem, and we’re excited to share all of the improvements we’re making across PubChem with you!

What’s working well? What’s not? What’s missing? Send an email to pubchem-help@ncbi.nlm.nih.gov

Stay tuned to this blog for future announcements about the roll-out of all the new designs and features coming to PubChem!

PubChem pages have a new look and feel!

We’ve redesigned PubChem’s summary and record pages with a series of updates both behind the scenes and to your own user experience. These changes will make finding the information you’re looking for easier and faster.

Behind the scenes, we’ve changed the way PubChem models and serves information. Most people won’t see these changes, but the changes do allow us to create pages better suited to your needs, faster. If you’re a programmatic user and you need information on how the data model changes affect you, you can find more information in our blog.

To go along with the behind the scenes changes, we’ve redesigned your user experience with an all new look and feel. To begin, we’ve started color-theming our pages to make it easier to recognize what kind of page you’re on. For example, compound pages may have a light blue theme, while substance pages a yellow theme. There will be themes for bioassay and other page types as well.

PubChem_Lindane_Compound_Page

PubChem Lindane Compound Summary Page

 

We’ve also looked over volumes of usage data and user feedback to improve page layout and navigation, and many of the changes you’ll see are a direct response to your feedback. For example, the table of contents has been moved and improved. It now appears on the right side of the page, and higher up on the page. Redundant navigational icons are being removed.

Another result of your feedback is that we’re emphasizing chemical safety information in the summary area at the top of the record. We’re also adding thumbnails for all the available structure types of a given compound (for example, 2D, 3D, crystal) to the summary area. In addition, the graphics quality of the compound 3D Conformer interactive model has been significantly improved.

Please note that, due to the number of upgrades we’re making to PubChem’s pages, legacy browsers will no longer be supported. You can find more information about browsers and browser support here:
https://www.ncbi.nlm.nih.gov/home/about/policies/#browsers

 

We want to know what you think!

PubChem’s new look and feel is a big step forward for PubChem, and we’re excited to share all of the improvements we’re making across PubChem with you!

What’s working well? What’s not? What’s missing? Send an email to pubchem-help@ncbi.nlm.nih.gov

Stay tuned to this blog for future announcements about the roll-out of all the new designs and features coming to PubChem!

Updates to the PubChem Data Model

PubChem is updating the data model for objects returned by the PUG View server. These objects are used by both programmatic users and by PubChem web pages. PubChem web users will not be directly affected by the data model changes. Programmatic users, however, will need to update the programs that retrieve and interpret data from PUG View. The following major changes are being made to the data model of the PUG View JSON/XML blobs:

  1. No more HTML markup within strings; instead, we will have an explicit markup object that separates primary strings from the various markup types.
  2. All values are lists, having separate fields for individual values.
  3. No more embedded tables in the data blobs.

 

No more HTML markup within strings.

PubChem is making a major effort to remove all embedded HTML from within the various strings in the data blobs. Such embedded markup is difficult for parsers to deal with when only a plain string is desired. For example, this is the old model:

{
    "StringValue": "Flipo RM: [Are the NSAIDs able to compromising the cardio-preventive efficacy of <a class=\"pubchem-internal-link CID-2244\" href=\"https://pubchem.ncbi.nlm.nih.gov/compound/aspirin\">aspirin</a>?]. Presse Med. 2006 Sep;35(9 Spec No 1):1S53-60.",
    "URL": "https://www.ncbi.nlm.nih.gov/pubmed/17078596"
}

In the new data model, the main string is in plain text, and the URL links (or other types of markup) are separate, with the character location of the markup on the original string indicated by start and length values. For example:

{
    "String": "Flipo RM: [Are the NSAIDs able to compromising the cardio-preventive efficacy of aspirin?]. Presse Med. 2006 Sep;35(9 Spec No 1):1S53-60. [PMID: 17078596]",
    "Markup": [
        {
            "Start": 139,
            "Length": 14,
            "URL": "https://www.ncbi.nlm.nih.gov/pubmed/17078596",
            "Type": "General link"
        },
        {
            "Start": 81,
            "Length": 7,
            "URL": "https://pubchem.ncbi.nlm.nih.gov/compound/aspirin",
            "Type": "PubChem Internal Link",
            "Extra": "CID-2244"
        }
    ]
}

This new format will make it easier for parsers to get at the relevant text data without a lot of programming overhead.  Please note that this removal of embedded HTML also includes escaped entities in HTML. These will instead be represented by a single UTF8 character (for example, “&deg;” à “°”) within the base string.

 

All values are lists.

In the new PubChem data model, all values are being converted to list types. We’ve done this to avoid the cumbersome necessity for data parsers to have to check separate fields for single values vs. lists. We’ll use JSON format for the following examples, but the XML data model is parallel to the JSON. Here are two examples of the old format:

{
    "ReferenceNumber": 19,
    "Name": "Melting Point",
    "Reference": [
        "PhysProp"
    ],
    "NumValue": 135,
    "ValueUnit": "°C"
}

or

"Information": [
    {
        "ReferenceNumber": 135,
        "Name": "Standard non-polar",
        "NumValueList": [
            1270,
            1315,
            1309,
            1309
        ]
    }
]

In the examples above, note the fields “NumValue” vs. “NumValueList” – this is cumbersome to code against. In the new system, the examples above would look like this, “Number” being used in both cases within a list structure:

{
    "ReferenceNumber": 35,
    "Name": "Melting Point",
    "Reference": [
        "PhysProp"
    ],
    "Value": {
        "Number": [
            135
        ],
    "Unit": "°C"
    }
}

or,

"Information": [
    {
        "ReferenceNumber": 75,
        "Name": "Standard non-polar",
        "Value": {
            "Number": [
                1270,
                1315,
                1309,
                1309
            ]
        }
    }
]

These changes will make it easier to code your data parsers.

 

No more embedded tables in the data blobs.

The use of embedded tables in the old system made it difficult for programmatic users to extract specific fields from within the table. The format required you to dig down into the rows and cells of the table to try and find the needed value. For example:

"Information": [
    {
        "ReferenceNumber": 182,
        "Name": "Computed Properties",
        "Table": {
            "ColumnName": [
                "Property Name",
                "Property Value"
            ],
            "Row": [
                {
                    "Cell": [
                        {
                            "StringValue": "Molecular Weight"
                        },
                        {
                            "NumValue": 180.159,
                            "ValueUnit": "g/mol"
                        }
                    ]
                }, …

In the new data below, the fields are more explicitly labeled with section names, the same way as other (non-table) values in the data:

{
    "TOCHeading": "Molecular Weight",
    "Description": "Molecular weight or molecular mass refers to the mass of a molecule. It is calculated as the sum of the mass of each constituent atom multiplied by the number of atoms of that element in the molecular formula.",
    "Information": [
        {
            "ReferenceNumber": 120,
            "Name": "Molecular Weight",
            "Value": {
                "Number": [
                    180.159
                ],
                "Unit": "g/mol"
            }
        }
    ]
}

These changes will make it easier to retrieve data from tables without a lot of programming overhead.

 

PubChem_PUG_View_Data_Model

PubChem PUG View Lindane Compound JSON data blob fragment

 

We want to know what you think!

In summary, PubChem’s new data model makes it easier to retrieve the data you need. As the data model is updated and released, you’ll be able to find detailed information on the schema here: https://pubchemdocs.ncbi.nlm.nih.gov/pug-view.

What’s working well? What’s not? What’s missing? Send an email to pubchem-help@ncbi.nlm.nih.gov

More than a million chemical-article links from Thieme Chemistry added into PubChem

More than a million links to scientific articles with a focus on chemical synthesis have been added to PubChem, thanks to contributions from the publisher Thieme Chemistry with support from their technology partner InfoChem. (Read Thieme’s press release about it.)

The Thieme Chemistry information in PubChem covers nearly 700,000 chemical substance records, nearly 700,000 scientific article descriptions, and over 1.2 million links between chemicals and articles.  The document descriptions include information such as a digital object identifier (DOI), publication title, name of the journal or book, publication type, language, and publication year.

The Thieme Chemistry contribution dramatically increases the number of chemical structures in PubChem with links to the scientific literature from nearly 1.0 million to 1.6 million.  Of the approximately 700,000 Thieme Chemistry chemical structures contributed to PubChem, 42% are new to PubChem, and 89% previously lacked literature links.

Finding chemicals with Thieme references

You can easily retrieve a complete list of PubChem Substance or Compound records that have Thieme references through the  PubChem Classification Browser, the PubChem Data Sources page, or by searching in the Substance or Compound databases.  (Not sure about how Compounds and Substances in PubChem are different from each other? Read this blog post.)

Within the PubChem Classification Browser, the PubChem Compound TOC (Table of Contents) classification tree allows you to find all chemicals with a given annotation section.  You can click “Literature” to view the subset fields under literature and find the “Thieme References” section.  Clicking on the number will then show compound records with that section.

The entire list of chemical substances provided by Thieme Chemistry is also available through the PubChem Data Sources page. (Read this blog post to learn more about the PubChem Data Sources page.)  Searching for “Thieme Chemistry” from the list of data sources shown on the page will lead you to the Thieme Chemistry data source page that has a link to the PubChem records provided by Thieme Chemistry.

You can also search the PubChem Compound or PubChem Substance databases directly using the query “Thieme Chemistry”[sourcename].

Thieme references in chemical records

Each chemical record with a Literature / Thieme References section includes a table containing document links from Thieme Chemistry.  The figure below shows the Thieme References section of the Compound record for ciprofloxacin (CID 2764).

The Literature / Thieme References section of the ciprofloxacin Compound record
Figure 1. The Literature / Thieme References section of the ciprofloxacin Compound record (CID 2764). Clicking the title (red circle) loads the article at the Thieme Chemistry site.

The article title links to the article on the Thieme-Chemistry site. You can download all references for a chemical record in CSV format through the “Download” button at the top right of the table. You can also expand to the full table by clicking the full screen icon, where you can see additional data columns.  By default, the articles are ordered by Publication Date as provided by Thieme Chemistry, but you can easily change the sorting order through the pulldown menu.

PubChem, along with contributors such as Thieme Chemistry, is helping to fuel a modern, data-driven research ecosystem.  Literature links from Thieme Chemistry dramatically expand the findability, accessibility, interoperability, and reusability (FAIR) of synthesis-related chemical information.  In addition, this contributed content helps to further enhance global open science by allowing researchers to locate key information about chemicals.