Gene¶
lamindb provides access to the following public gene ontologies through bionty:
Here we show how to access and search gene ontologies to standardize new data.
Setup¶
!lamin init --storage ./test-public-ontologies --schema bionty
💡 connected lamindb: testuser1/test-public-ontologies
import bionty as bt
import pandas as pd
💡 connected lamindb: testuser1/test-public-ontologies
PublicOntology objects¶
Let us create a public ontology accessor with public(), which chooses a default public ontology source from PublicSource. It’s a PublicOntology object, which you can think about as a public registry:
public = bt.Gene.public(organism="human")
public
PublicOntology
Entity: Gene
Organism: human
Source: ensembl, release-112
#terms: 75829
As for registries, you can export the ontology as a DataFrame:
df = public.df()
df.head()
| ensembl_gene_id | symbol | ncbi_gene_id | biotype | description | synonyms | |
|---|---|---|---|---|---|---|
| 0 | ENSG00000000003 | TSPAN6 | 7105 | protein_coding | tetraspanin 6 | TSPAN-6|T245|TM4SF6 |
| 1 | ENSG00000000005 | TNMD | 64102 | protein_coding | tenomodulin | TEM|MYODULIN|CHM1L|TENDIN|BRICD4 |
| 2 | ENSG00000000419 | DPM1 | 8813 | protein_coding | dolichyl-phosphate mannosyltransferase subunit... | CDGIE|MPDS |
| 3 | ENSG00000000457 | SCYL3 | 57147 | protein_coding | SCY1 like pseudokinase 3 | PACE1|PACE-1 |
| 4 | ENSG00000000460 | FIRRM | 55732 | protein_coding | FIGNL1 interacting regulator of recombination ... | C1ORF112|FLJ10706|APOLO1|FLIP|MEICA1 |
Unlike registries, you can also export it as a Pronto object via public.ontology.
Look up terms¶
As for registries, terms can be looked up with auto-complete:
lookup = public.lookup()
The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):
lookup.tcf7
Gene(ensembl_gene_id='ENSG00000081059', symbol='TCF7', ncbi_gene_id='6932', biotype='protein_coding', description='transcription factor 7 ', synonyms='TCF-1')
To look up the exact original strings, convert the lookup object to dict and use the [] accessor:
lookup_dict = lookup.dict()
lookup_dict["TCF7"]
Gene(ensembl_gene_id='ENSG00000081059', symbol='TCF7', ncbi_gene_id='6932', biotype='protein_coding', description='transcription factor 7 ', synonyms='TCF-1')
By default, the name field is used to generate lookup keys. You can specify another field to look up:
lookup = public.lookup(public.ncbi_gene_id)
If multiple entries are matched, they are returned as a list:
lookup.bt_100126572
Gene(ensembl_gene_id='ENSG00000203733', symbol='GJE1', ncbi_gene_id='100126572', biotype='protein_coding', description='gap junction protein epsilon 1 ', synonyms='CX23')
Search terms¶
Search behaves in the same way as it does for registries:
public.search("TP53").head(3)
| ensembl_gene_id | ncbi_gene_id | biotype | description | synonyms | __ratio__ | |
|---|---|---|---|---|---|---|
| symbol | ||||||
| TP53 | ENSG00000141510 | 7157 | protein_coding | tumor protein p53 | LFS1|P53 | 100.0 |
| TP53TG3D | ENSG00000205456 | 102723655 | protein_coding | TP53 target 3D | 90.0 | |
| TP53TG3C | ENSG00000205457 | 24150 | protein_coding | TP53 target 3C | 90.0 |
By default, search also covers synonyms:
public.search("PDL1").head(3)
| ensembl_gene_id | ncbi_gene_id | biotype | description | synonyms | __ratio__ | |
|---|---|---|---|---|---|---|
| symbol | ||||||
| CD274 | ENSG00000120217 | 29126 | protein_coding | CD274 molecule | PD-L1|PDCD1LG1|B7H1|PDL1|B7-H1|B7-H | 100.0 |
| GAPDHP69 | ENSG00000223460 | None | processed_pseudogene | glyceraldehyde 3 phosphate dehydrogenase pseud... | GAPDL14|GAPDHL14 | 90.0 |
| GAPDHP68 | ENSG00000233876 | None | processed_pseudogene | glyceraldehyde 3 phosphate dehydrogenase pseud... | GAPDHL13|GAPDL13 | 90.0 |
You can turn this off synonym by passing synonyms_field=None:
public.search("PDL1", synonyms_field=None).head(3)
| ensembl_gene_id | ncbi_gene_id | biotype | description | synonyms | __ratio__ | |
|---|---|---|---|---|---|---|
| symbol | ||||||
| SPDL1 | ENSG00000040275 | 54908 | protein_coding | spindle apparatus coiled-coil protein 1 | CCDC99|HSPINDLY|FLJ20364 | 88.888889 |
| PODNL1 | ENSG00000132000 | 79883 | protein_coding | podocan like 1 | SLRR5B|FLJ23447 | 80.000000 |
| PKD2L1 | ENSG00000107593 | 9033 | protein_coding | polycystin 2 like 1, transient receptor potent... | PKD2L|PCL|PKDL|TRPP3 | 80.000000 |
Search another field (default is .name):
public.search("tumor protein p53", field=public.description).head()
| ensembl_gene_id | symbol | ncbi_gene_id | biotype | synonyms | __ratio__ | |
|---|---|---|---|---|---|---|
| description | ||||||
| tumor protein p53 | ENSG00000141510 | TP53 | 7157 | protein_coding | LFS1|P53 | 100.000000 |
| tumor protein p73 | ENSG00000078900 | TP73 | 7161 | protein_coding | P73 | 94.117647 |
| tumor protein p63 | ENSG00000073282 | TP63 | 8626 | protein_coding | TP53CP|TP73L|P73L|OFC8|EEC3|P51|P53CP|SHFM4|TP... | 94.117647 |
| tumor protein D52 | ENSG00000076554 | TPD52 | 124188259 | protein_coding | D52|HD52|N8L | 88.235294 |
| tumor protein D52 | ENSG00000076554 | TPD52 | 7163 | protein_coding | D52|HD52|N8L | 88.235294 |
Standardize gene identifiers¶
Let us generate a DataFrame that stores a number of gene identifiers, some of which corrupted:
data = {
"gene symbol": ["A1CF", "A1BG", "FANCD1", "corrupted"],
"ncbi id": ["29974", "1", "5133", "corrupted"],
"ensembl_gene_id": [
"ENSG00000148584",
"ENSG00000121410",
"ENSG00000188389",
"ENSGcorrupted",
],
}
df_orig = pd.DataFrame(data).set_index("ensembl_gene_id")
df_orig
| gene symbol | ncbi id | |
|---|---|---|
| ensembl_gene_id | ||
| ENSG00000148584 | A1CF | 29974 |
| ENSG00000121410 | A1BG | 1 |
| ENSG00000188389 | FANCD1 | 5133 |
| ENSGcorrupted | corrupted | corrupted |
First we can check whether any of our values are validated against the ontology reference:
validated = public.validate(df_orig.index, public.ensembl_gene_id)
df_orig.index[~validated]
✅ 3 terms (75.00%) are validated
❗ 1 term (25.00%) is not validated: ENSGcorrupted
Index(['ENSGcorrupted'], dtype='object', name='ensembl_gene_id')
Next, we validate which symbols are mappable against the ontology:
# based on NCBI gene ID
public.validate(df_orig["ncbi id"], public.ncbi_gene_id)
✅ 3 terms (75.00%) are validated
❗ 1 term (25.00%) is not validated: corrupted
array([ True, True, True, False])
# based on Gene symbols
validated_symbols = public.validate(df_orig["gene symbol"], public.symbol)
df_orig["gene symbol"][~validated_symbols]
✅ 2 terms (50.00%) are validated
❗ 2 terms (50.00%) are not validated: FANCD1, corrupted
ensembl_gene_id
ENSG00000188389 FANCD1
ENSGcorrupted corrupted
Name: gene symbol, dtype: object
Here, 2 of the gene symbols are not validated. Inspect why:
public.inspect(df_orig["gene symbol"], public.symbol);
✅ 2 terms (50.00%) are validated for symbol
❗ 2 terms (50.00%) are not validated for symbol: FANCD1, corrupted
detected 1 terms with synonym: FANCD1
→ standardize terms via .standardize()
Logging suggests to use .standardize():
mapped_symbol_synonyms = public.standardize(df_orig["gene symbol"])
mapped_symbol_synonyms
💡 standardized 3/4 terms
['A1CF', 'A1BG', 'BRCA2', 'corrupted']
Optionally, you can return a mapper in the form of {synonym1: standardized_name1, ...}:
public.standardize(df_orig["gene symbol"], return_mapper=True)
💡 standardized 3/4 terms
{'FANCD1': 'BRCA2'}
We can use the standardized symbols as the new standardized index:
df_curated = df_orig.reset_index()
df_curated.index = mapped_symbol_synonyms
df_curated
| ensembl_gene_id | gene symbol | ncbi id | |
|---|---|---|---|
| A1CF | ENSG00000148584 | A1CF | 29974 |
| A1BG | ENSG00000121410 | A1BG | 1 |
| BRCA2 | ENSG00000188389 | FANCD1 | 5133 |
| corrupted | ENSGcorrupted | corrupted | corrupted |
You can convert identifiers by passing return_field to standardize():
public.standardize(
df_curated.index,
field=public.symbol,
return_field=public.ensembl_gene_id,
)
💡 standardized 3/4 terms
['ENSG00000148584', 'ENSG00000121410', 'ENSG00000139618', 'corrupted']
And return mappable identifiers as a dict:
public.standardize(
df_curated.index,
field=public.symbol,
return_field=public.ensembl_gene_id,
return_mapper=True,
)
💡 standardized 3/4 terms
{'A1BG': 'ENSG00000121410',
'BRCA2': 'ENSG00000139618',
'A1CF': 'ENSG00000148584'}
Ontology source versions¶
For any given entity, we can choose from a number of versions:
bt.PublicSource.filter(entity="Gene").df()
| uid | entity | organism | currently_used | source | source_name | version | url | md5 | source_website | run_id | created_by_id | updated_at | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||
| 11 | 1hx4 | Gene | human | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_human__ensembl__release-... | 4ccda4d88720a326737376c534e8446b | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484524+00:00 |
| 12 | 4Wsn | Gene | human | False | ensembl | Ensembl | release-111 | s3://bionty-assets/df_human__ensembl__release-... | f9183bc44abb34459984e137b5de8af1 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484618+00:00 |
| 13 | 4yVc | Gene | human | False | ensembl | Ensembl | release-110 | s3://bionty-assets/df_human__ensembl__release-... | 832f3947e83664588d419608a469b528 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484711+00:00 |
| 14 | 4dlM | Gene | human | False | ensembl | Ensembl | release-109 | s3://bionty-assets/human_ensembl_release-109_G... | 72da9968c74e96d136a489a6102a4546 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484803+00:00 |
| 15 | 76FX | Gene | mouse | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_mouse__ensembl__release-... | 519cf7b8acc3c948274f66f3155a3210 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484895+00:00 |
| 16 | s5S2 | Gene | mouse | False | ensembl | Ensembl | release-111 | s3://bionty-assets/df_mouse__ensembl__release-... | 5c071655347458307ac92b208f3c903a | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484988+00:00 |
| 17 | 2akp | Gene | mouse | False | ensembl | Ensembl | release-110 | s3://bionty-assets/df_mouse__ensembl__release-... | fa4ce130f2929aefd7ac3bc8eaf0c4de | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.485083+00:00 |
| 18 | 3JQx | Gene | mouse | False | ensembl | Ensembl | release-109 | s3://bionty-assets/mouse_ensembl_release-109_G... | 08a1165061151b270b985317322bd2ed | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.485177+00:00 |
| 19 | 7LW6 | Gene | saccharomyces cerevisiae | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_saccharomyces cerevisiae... | 11775126b101233525a0a9e2dd64edae | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.485271+00:00 |
| 20 | 1HSy | Gene | saccharomyces cerevisiae | False | ensembl | Ensembl | release-111 | s3://bionty-assets/df_saccharomyces cerevisiae... | a15fab1d9d15a56d32fd2fd8a8fa250a | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.485363+00:00 |
| 21 | 2UvD | Gene | saccharomyces cerevisiae | False | ensembl | Ensembl | release-110 | s3://bionty-assets/df_saccharomyces cerevisiae... | 2e59495a3e87ea6575e408697dd73459 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.485456+00:00 |
When instantiating a Bionty object, we can choose a source or version:
public_source = bt.PublicSource.filter(
source="ensembl", version="release-110", organism="human"
).one()
public = bt.Gene.public(public_source=public_source)
public
❗ loading non-default source inside a LaminDB instance
PublicOntology
Entity: Gene
Organism: human
Source: ensembl, release-110
#terms: 75719
The currently used ontologies can be displayed using:
bt.PublicSource.filter(currently_used=True).df()
Show code cell output
| uid | entity | organism | currently_used | source | source_name | version | url | md5 | source_website | run_id | created_by_id | updated_at | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||
| 1 | 5Dlc | Organism | vertebrates | True | ensembl | Ensembl | release-112 | https://ftp.ensembl.org/pub/release-112/specie... | 0ec37e77f4bc2d0b0b47c6c62b9f122d | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.483555+00:00 |
| 6 | 2Jzh | Organism | bacteria | True | ensembl | Ensembl | release-57 | https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... | ee28510ed5586ea7ab4495717c96efc8 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484046+00:00 |
| 7 | 1kdI | Organism | fungi | True | ensembl | Ensembl | release-57 | http://ftp.ensemblgenomes.org/pub/fungi/releas... | dbcde58f4396ab8b2480f7fe9f83df8a | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484138+00:00 |
| 8 | 2mIM | Organism | metazoa | True | ensembl | Ensembl | release-57 | http://ftp.ensemblgenomes.org/pub/metazoa/rele... | 424636a574fec078a61cbdddb05f9132 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484233+00:00 |
| 9 | 2XQ6 | Organism | plants | True | ensembl | Ensembl | release-57 | https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... | eadaa1f3e527e4c3940c90c7fa5c8bf4 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484333+00:00 |
| 10 | 1Vzs | Organism | all | True | ncbitaxon | NCBItaxon Ontology | 2023-06-20 | s3://bionty-assets/df_all__ncbitaxon__2023-06-... | 00d97ba65627f1cd65636d2df22ea76c | https://github.com/obophenotype/ncbitaxon | None | 1 | 2024-05-29 09:56:16.484429+00:00 |
| 11 | 1hx4 | Gene | human | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_human__ensembl__release-... | 4ccda4d88720a326737376c534e8446b | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484524+00:00 |
| 15 | 76FX | Gene | mouse | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_mouse__ensembl__release-... | 519cf7b8acc3c948274f66f3155a3210 | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.484895+00:00 |
| 19 | 7LW6 | Gene | saccharomyces cerevisiae | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_saccharomyces cerevisiae... | 11775126b101233525a0a9e2dd64edae | https://www.ensembl.org | None | 1 | 2024-05-29 09:56:16.485271+00:00 |
| 22 | 7llW | Protein | human | True | uniprot | Uniprot | 2023-03 | s3://bionty-assets/df_human__uniprot__2023-03_... | 1c46e85c6faf5eff3de5b4e1e4edc4d3 | https://www.uniprot.org | None | 1 | 2024-05-29 09:56:16.485547+00:00 |
| 24 | 5U7J | Protein | mouse | True | uniprot | Uniprot | 2023-03 | s3://bionty-assets/df_mouse__uniprot__2023-03_... | 9d5e9a8225011d3218e10f9bbb96a46c | https://www.uniprot.org | None | 1 | 2024-05-29 09:56:16.485730+00:00 |
| 26 | 5nkB | CellMarker | human | True | cellmarker | CellMarker | 2.0 | s3://bionty-assets/human_cellmarker_2.0_CellMa... | d565d4a542a5c7e7a06255975358e4f4 | http://bio-bigdata.hrbmu.edu.cn/CellMarker | None | 1 | 2024-05-29 09:56:16.485913+00:00 |
| 27 | 6AFz | CellMarker | mouse | True | cellmarker | CellMarker | 2.0 | s3://bionty-assets/mouse_cellmarker_2.0_CellMa... | 189586732c63be949e40dfa6a3636105 | http://bio-bigdata.hrbmu.edu.cn/CellMarker | None | 1 | 2024-05-29 09:56:16.486005+00:00 |
| 28 | 6cbC | CellLine | all | True | clo | Cell Line Ontology | 2022-03-21 | https://data.bioontology.org/ontologies/CLO/su... | ea58a1010b7e745702a8397a526b3a33 | https://bioportal.bioontology.org/ontologies/CLO | None | 1 | 2024-05-29 09:56:16.486097+00:00 |
| 29 | 3DeN | CellType | all | True | cl | Cell Ontology | 2024-02-13 | http://purl.obolibrary.org/obo/cl/releases/202... | https://obophenotype.github.io/cell-ontology | None | 1 | 2024-05-29 09:56:16.486189+00:00 | |
| 34 | 1AyH | Tissue | all | True | uberon | Uberon multi-species anatomy ontology | 2024-02-20 | http://purl.obolibrary.org/obo/uberon/releases... | 2048667b5fdf93192384bdf53cafba18 | http://obophenotype.github.io/uberon | None | 1 | 2024-05-29 09:56:16.486674+00:00 |
| 39 | LoCG | Disease | all | True | mondo | Mondo Disease Ontology | 2024-02-06 | http://purl.obolibrary.org/obo/mondo/releases/... | 78914fa236773c5ea6605f7570df6245 | https://mondo.monarchinitiative.org | None | 1 | 2024-05-29 09:56:16.487135+00:00 |
| 44 | 2mou | Disease | human | True | doid | Human Disease Ontology | 2024-01-31 | http://purl.obolibrary.org/obo/doid/releases/2... | b36c15a4610757094f8db64b78ae2693 | https://disease-ontology.org | None | 1 | 2024-05-29 09:56:16.487603+00:00 |
| 51 | 4usY | ExperimentalFactor | all | True | efo | The Experimental Factor Ontology | 3.63.0 | http://www.ebi.ac.uk/efo/releases/v3.63.0/efo.owl | 603e6f6981d53d501c5921aa3940b095 | https://bioportal.bioontology.org/ontologies/EFO | None | 1 | 2024-05-29 09:56:16.488253+00:00 |
| 54 | 2WLc | Phenotype | human | True | hp | Human Phenotype Ontology | 2024-03-06 | https://github.com/obophenotype/human-phenotyp... | 36b0d00c24a68edb9131707bc146a4c7 | https://hpo.jax.org | None | 1 | 2024-05-29 09:56:16.488529+00:00 |
| 58 | 6zE1 | Phenotype | mammalian | True | mp | Mammalian Phenotype Ontology | 2024-02-07 | https://github.com/mgijax/mammalian-phenotype-... | 31c27ed2c7d5774f8b20a77e4e1fd278 | https://github.com/mgijax/mammalian-phenotype-... | None | 1 | 2024-05-29 09:56:16.488896+00:00 |
| 60 | 7EnA | Phenotype | zebrafish | True | zp | Zebrafish Phenotype Ontology | 2024-01-22 | https://github.com/obophenotype/zebrafish-phen... | 01600a5d392419b27fc567362d4cfff8 | https://github.com/obophenotype/zebrafish-phen... | None | 1 | 2024-05-29 09:56:16.489080+00:00 |
| 63 | 55lY | Phenotype | all | True | pato | Phenotype And Trait Ontology | 2023-05-18 | http://purl.obolibrary.org/obo/pato/releases/2... | bd472f4971492109493d4ad8a779a8dd | https://github.com/pato-ontology/pato | None | 1 | 2024-05-29 09:56:16.489354+00:00 |
| 64 | 48aa | Pathway | all | True | go | Gene Ontology | 2023-05-10 | https://data.bioontology.org/ontologies/GO/sub... | e9845499eadaef2418f464cd7e9ac92e | http://geneontology.org | None | 1 | 2024-05-29 09:56:16.489445+00:00 |
| 67 | 3rm9 | BFXPipeline | all | True | lamin | Bioinformatics Pipeline | 1.0.0 | s3://bionty-assets/bfxpipelines.json | a7eff57a256994692fba46e0199ffc94 | https://lamin.ai | None | 1 | 2024-05-29 09:56:16.489724+00:00 |
| 68 | 5alK | Drug | all | True | dron | Drug Ontology | 2024-03-02 | https://data.bioontology.org/ontologies/DRON/s... | 84138459de4f65034e979f4e46783747 | https://bioportal.bioontology.org/ontologies/DRON | None | 1 | 2024-05-29 09:56:16.489824+00:00 |
| 70 | 7CRn | DevelopmentalStage | human | True | hsapdv | Human Developmental Stages | 2020-03-10 | http://aber-owl.net/media/ontologies/HSAPDV/11... | 52181d59df84578ed69214a5cb614036 | https://github.com/obophenotype/developmental-... | None | 1 | 2024-05-29 09:56:16.490012+00:00 |
| 71 | 16tR | DevelopmentalStage | mouse | True | mmusdv | Mouse Developmental Stages | 2020-03-10 | http://aber-owl.net/media/ontologies/MMUSDV/9/... | 5bef72395d853c7f65450e6c2a1fc653 | https://github.com/obophenotype/developmental-... | None | 1 | 2024-05-29 09:56:16.490106+00:00 |
| 72 | 3Tlc | Ethnicity | human | True | hancestro | Human Ancestry Ontology | 3.0 | https://github.com/EBISPOT/hancestro/raw/3.0/h... | 76dd9efda9c2abd4bc32fc57c0b755dd | https://github.com/EBISPOT/hancestro | None | 1 | 2024-05-29 09:56:16.492428+00:00 |
| 73 | 5JnV | BioSample | all | True | ncbi | NCBI BioSample attributes | 2023-09 | s3://bionty-assets/df_all__ncbi__2023-09__BioS... | 918db9bd1734b97c596c67d9654a4126 | https://www.ncbi.nlm.nih.gov/biosample/docs/at... | None | 1 | 2024-05-29 09:56:16.492535+00:00 |