F# Data: Making structured data first class citizens

•Download as PPTX, PDF•

6 likes•1,672 views

Slides from a talk at DCP 2014 workshop, about F# Data type providers. For live code samples, see the YouTube links in the slides!

Technology Education

F# Data
Making structured data first-class citizens
Tomas Petricek, University of Cambridge
Project homepage: http://fsharp.github.io/FSharp.Data
Get in touch: @tomaspetricek | tomas@tomasp.net

F# Software Foundation
http://www.fsharp.org
software stacks
trainings teaching F# user groups snippets
mac and linux cross-platform tutorials
F# community open-source MonoDevelop
contributions research support
consultancy mailing list

F# Data type providers
First-class data
CSV, REST, WorldBank…
R Type provider
Statistics & visualization
5000 tested packages
www.fslab.org

Deedle data frame
Data exploration
Indexing and aggregation
F# Charting library
Simple & composable
Interactive style
www.fslab.org

What are type providers?
Integrating WorldBank and R
http://youtu.be/7r2-B-5H_io

Type provider research questions
Data vs.
Schema
Laziness
Mapping
to types
Schema
inference
Schema
inference

Schema inference
Loading Titanic data from CSV
http://youtu.be/yjBdZduc0ko

Inferring primitive types
null intbool
string
decimal
float

Structure inference
Working with XML and JSON data
http://youtu.be/_DjX0ybaXZY
http://youtu.be/SkZBzlREOMo

Inferring structured types
person { name : string } person { name : string, age : int }
person { name : string, age : int option }
[ { num : int } ] [ { str : string } ]
[ { num : int option, str : string option } ]
int { value : int }
int + { value : int }

Does it scale?
Query movies using Apiary provider
http://youtu.be/-Am2uRUv39c

Conclusions
Inference from small-scale samples works!
Schema is (very) often missing
But data is (very) often regular
Check out F# Data and contribute!
Project homepage: http://fsharp.github.io/FSharp.Data
Get in touch: @tomaspetricek | tomas@tomasp.net

What's hot

Benchmark BIB-R @ TPDL 2016

Joffrey DECOURSELLE

Summary of crowdsourcing studies to assess the quality of knowledge graphs and complete missing values. Results focus on findings over the DBpedia knowledge graph ( https://wiki.dbpedia.org/). Related publications: Acosta, M., Zaveri, A., Simperl, E., Kontokostas, D., Auer, S., & Lehmann, J. Crowdsourcing Linked Data Quality Assessment. In International Semantic Web Conference (pp. 260-276), 2013. Acosta, M., Zaveri, A., Simperl, E., Kontokostas, D., Flöck, F., & Lehmann, J. Detecting Linked Data Quality issues via Crowdsourcing: A DBpedia Study. Semantic Web Journal, 9(3), 303-335, 2018. Acosta, M., Simperl, E., Flöck, F., & Vidal, M. E. HARE: A hybrid SPARQL engine to enhance query answers via crowdsourcing. In Proceedings of the 8th International Conference on Knowledge Capture (p. 11). 2015. Best Student Paper Award. Acosta, M., Simperl, E., Flöck, F., & Vidal, M. E. Enhancing answer completeness of SPARQL queries via crowdsourcing. Journal of Web Semantics, 45, 41-62, 2017. Acosta, M., Simperl, E., Flöck, F., & Vidal, M. E. HARE: An engine for enhancing answer completeness of SPARQL queries via crowdsourcing. Companion Volume of the Web Conference (pp. 501-505). 2018.

Crowdsourcing the Quality of Knowledge Graphs:A DBpedia Study

Maribel Acosta Deibe

Newspapers, IIIF, and ALTO

Karen Estlund

A Closer Look at the Changing Dynamics of DBpedia Mappings

Maribel Acosta Deibe

Milex 2010 final

Albert S. Cook Library

Topical_Facets

Eric Van Horenbeeck

Creating a FRBRized Data Structure in XML

Jenn Riley

This is the presentation from the (meta)-data quality workshop at TPDL 2017 in Thessaloniki, Greece. Europeana.eu aggregates metadata describing more than 55 million cultural heritage objects from libraries, museums, archives and audiovisual archives across Europe. Quality (and particularly multilingual quality) of metadata is crucial for enabling search and data re-use in digital libraries. Capturing multilingual aspects of the data requires us to take into account the full lifecycle of data aggregation including data enhancement processes such as data enrichment. Multilinguality cannot be captured as one measure, but needs to be considered as an intersection of several measures, bringing challenges when interpreting and visualising the results. This presentation presents an approach for capturing multilinguality as part of data quality dimensions, namely completeness, consistency and accessibility.

Data Quality Assessment in Europeana: Metrics for Multilinguality

Juliane Stiller

Search4similars

Valentin Bazarevsky

Analyzing poetry databases to develop a metadata application profile. Why eac...

Uned Laboratorio de Innovación en Humanidades

Knowledge graphs have become popular over the past decade and frequently rely on the Resource Description Framework (RDF) or property graph databases as data models. We present, the first translator from SPARQL -- the W3C standardised language for RDF -- and Gremlin -- a popular property graph traversal language. Gremlinator translates SPARQL queries to Gremlin path traversals for executing graph pattern matching queries over graph databases. This allows a user, who is well versed in SPARQL, to access and query a wide variety of Graph Data Management Systems (DMSs) avoiding the steep learning curve for adapting to a new Graph Query Language (GQL). Gremlin is a graph computing system agnostic traversal language (covering both OLTP graph database or OLAP graph processors), making it a desirable choice for supporting interoperability for querying Graph DMSs.

Sparql querying of-property-graphs-harsh thakkar-graph day 2017 sf

Harsh Thakkar

Oles Petriv “Creating one concept embedding space for persons, brands and new...

Lviv Startup Club

Link Discovery Tutorial Introduction

Holistic Benchmarking of Big Linked Data

Entity Retrieval (WSDM 2014 tutorial)

krisztianbalog

Entity Retrieval (SIGIR 2013 tutorial)

krisztianbalog

AjayBhullar_Resume (5)

Ajay Bhullar

Healthcare Data Management using Domain Specific Languages for Metadata Manag...

David Milward

Summary of GSCL 2013 international NLP conference in Germany

Lifeng (Aaron) Han

Linked Data APIs (Funding Circle May 2015)

Antonio Garrote Hernández

Session 1.6 fostering interoperability of european qualifications: the qual...

semanticsconference

What's hot (20)

Benchmark BIB-R @ TPDL 2016

Crowdsourcing the Quality of Knowledge Graphs:A DBpedia Study

Newspapers, IIIF, and ALTO

A Closer Look at the Changing Dynamics of DBpedia Mappings

Milex 2010 final

Topical_Facets

Creating a FRBRized Data Structure in XML

Data Quality Assessment in Europeana: Metrics for Multilinguality

Search4similars

Analyzing poetry databases to develop a metadata application profile. Why eac...

Sparql querying of-property-graphs-harsh thakkar-graph day 2017 sf

Oles Petriv “Creating one concept embedding space for persons, brands and new...

Link Discovery Tutorial Introduction

Entity Retrieval (WSDM 2014 tutorial)

Entity Retrieval (SIGIR 2013 tutorial)

AjayBhullar_Resume (5)

Healthcare Data Management using Domain Specific Languages for Metadata Manag...

Summary of GSCL 2013 international NLP conference in Germany

Linked Data APIs (Funding Circle May 2015)

Session 1.6 fostering interoperability of european qualifications: the qual...

Similar to F# Data: Making structured data first class citizens

F# and Financial Data Making Data Analysis Simple

Tomas Petricek

IIIF for CNI Spring 2014 Membership Meeting

Tom-Cramer

Wimmics Overview 2021

Fabien Gandon

The presentation investigates the challenges that we must face to share scientific datasets on the Web following the Linked Open Data principles. We present the standards of the Semantic Web and investigate how they can help address those challenges. We give tips as to how to choose vocabularies to describe data and metadata, link datasets to other related datasets by making appropriate alignments, translate existing data sources to RDF and publish it on the Web as linked data.

Make our Scientific Datasets Accessible and Interoperable on the Web

Franck Michel

Data Portability with SIOC and FOAF

Uldis Bojars

EDF2012 Mariana Damova - Factforge

European Data Forum

Linked Open Data

Laura Hollink

Semantic Pipes and Semantic Mashups

giurca

Publishing data on the Semantic Web

Peter Mika

In the media industry there is a great emphasis on providing descriptive metadata as part of the media assets to the consumers. Information extraction (IE) is considered an important tool for metadata generation process and its performance largely depend on the knowledge base it utilizes. The advances in the “Linked Data Cloud” research provide a great opportunity for generating such knowledge base that benefit from the participation of wider community. In this talk, I will discuss our experiences of utilizing Linked Data Cloud in conjunction with a GATE-based IE system.

Information Extraction and Linked Data Cloud

Dhaval Thakker

NISO Webinar: Experimenting with BIBFRAME: Reports from Early Adopters About the Webinar In May 2011, the Library of Congress officially launched a new modeling initiative, Bibliographic Framework Initiative, as a linked data alternative to MARC. The Library then announced in November 2012 the proposed model, called BIBFRAME. Since then, the library world is moving from mainly theorizing about the BIBFRAME model to attempts to implement practical experimentation and testing. This experimentation is iterative, and continues to shape the model so that it’s stable enough and broadly acceptable enough for adoption. In this webinar, several institutions will share their progress in experimenting with BIBFRAME within their library system. They will discuss the existing, developing, and planned projects happening at their institutions. Challenges and opportunities in exploring and implementing BIBFRAME in their institutions will be discussed as well. Agenda Introduction Todd Carpenter, Executive Director, NISO Experimental Mode: The National Library of Medicine and experiences with BIBFRAME Nancy Fallgren, Metadata Specialist Librarian, National Library of Medicine, National Institutes of Health, US Department of Health and Human Services (DHHS) Exploring BIBFRAME at a Small Academic Library Jeremy Nelson, Metadata and Systems Librarian, Colorado College Working with BIBFRAME for discovery and production: Linked data for Libraries/Linked Data for Production Nancy Lorimer, Head, Metadata Dept, Stanford University Libraries

April 8 NISO Webinar: Experimenting with BIBFRAME: Reports from Early Adopters

National Information Standards Organization (NISO)

How the Web can change social science research (including yours)

Frank van Harmelen

A Semantic Multimedia Web (Part 3)

Raphael Troncy

Linking library data

Jindřich Mynarz

Ranking Resources in Folksonomies by Exploiting Semantic and Context-specific...

Thomas Rodenhausen

Open Data Mashups: linking fragments into mosaics

phduchesne

Semantic Search Summer School2009

Peter Mika

The Semantic Data Web, Sören Auer, University of Leipzig

LOD2 Creating Knowledge out of Interlinked Data

Over the last years, the Semantic Web has been growing steadily. Today, we count more than 10,000 datasets made available online following Semantic Web standards. Nevertheless, many applications, such as data integration, search, and interlinking, may not take the full advantage of the data without having a priori statistical information about its internal structure and coverage. In fact, there are already a number of tools, which offer such statistics, providing basic information about RDF datasets and vocabularies. However, those usually show severe deficiencies in terms of performance once the dataset size grows beyond the capabilities of a single machine. In this paper, we introduce a software component for statistical calculations of large RDF datasets, which scales out to clusters of machines. More specifically, we describe the first distributed inmemory approach for computing 32 different statistical criteria for RDF datasets using Apache Spark. The preliminary results show that our distributed approach improves upon a previous centralized approach we compare against and provides approximately linear horizontal scale-up. The criteria are extensible beyond the 32 default criteria, is integrated into the larger SANSA framework and employed in at least four major usage scenarios beyond the SANSA community.

DistLODStats: Distributed Computation of RDF Dataset Statistics - ISWC 2018 talk

Gezim Sejdiu

Methodological Guidelines for Publishing Linked Data

Boris Villazón-Terrazas

Similar to F# Data: Making structured data first class citizens (20)

F# and Financial Data Making Data Analysis Simple

IIIF for CNI Spring 2014 Membership Meeting

Wimmics Overview 2021

Make our Scientific Datasets Accessible and Interoperable on the Web

Data Portability with SIOC and FOAF

EDF2012 Mariana Damova - Factforge

Linked Open Data

Semantic Pipes and Semantic Mashups

Publishing data on the Semantic Web

Information Extraction and Linked Data Cloud

April 8 NISO Webinar: Experimenting with BIBFRAME: Reports from Early Adopters

How the Web can change social science research (including yours)

A Semantic Multimedia Web (Part 3)

Linking library data

Ranking Resources in Folksonomies by Exploiting Semantic and Context-specific...

Open Data Mashups: linking fragments into mosaics

Semantic Search Summer School2009

The Semantic Data Web, Sören Auer, University of Leipzig

DistLODStats: Distributed Computation of RDF Dataset Statistics - ISWC 2018 talk

Methodological Guidelines for Publishing Linked Data

More from Tomas Petricek

Coeffects: A Calculus of Context-Dependent Computation

Tomas Petricek

When designing software, we often need to solve numerous instances of the same problem. When designing user-interfaces, we need to describe the layout and interaction. When financial systems, we need to describe a wide range of financial contracts or, for example, recognize different patterns in price change. Domain Specific Languages (DSLs) give us a way to solve such repeating problems. By designing a composable functional library, we can build an expressive language for describing our problems. Using the flexible F# syntax, we can get code that even non-programmers can understand. In this practically oriented talk, we'll develop a number of sample DSLs, ending with realistic examples. We'll cover both basic principles of DSL design as well as advanced tricks, such as using the F# 3.0 query syntax.

Creating Domain Specific Languages in F#

Tomas Petricek

F# Type Providers in Depth

Tomas Petricek

Asynchronous programming in F# (QCon 2012)

Tomas Petricek

Queries in general purpose languages

Tomas Petricek

Docase notation for Haskell

Tomas Petricek

Slides from my talk at GOTO 2011. The presentation discusses how to bridge the impedance mismatch between the structure used in programming languages (classes, records, unions, ...) and structure used in data sources (database schema, XML file or JSON document). The talk shows how to solve this problem at three scales - using dynamic typing, by defining structure explicitly and using F# type providers.

Accessing loosely structured data from F# and C#

Tomas Petricek

F# on the Server-Side

Tomas Petricek

F# Tutorial @ QCon

Tomas Petricek

One of the key concepts of functional programming is compositionality – the fact that we can build complex software using several basic “building blocks” and just a few ways of combining them. In this talk, I’ll demonstrate how we can teach this concept using F#. We’ll start with a basic example of this approach that even high-school students can easily understand – numerical expressions. Then we’ll move to more exciting examples of using library for composing 3D graphics. We’ll see that creating a fractal tree is not much more complicated than calculating the area of a triangle. The recording is available online at: http://www.communityforfsharp.net/

Teaching F#

Tomas Petricek

F# in MonoDevelop

Tomas Petricek

Academia

Tomas Petricek

Talk about message passing concurrency in F# using agents. In this style of programming, we write application as a group of agents that run in parallel and communicate by sending immutable messages to each other. We’ll discuss several common patterns that you can use for structuring agent-based applications. For example, we’ll look how to implement pipeline processing and map-reduce pattern using agents.

Concurrent programming with Agents

Tomas Petricek

More from Tomas Petricek (13)

Coeffects: A Calculus of Context-Dependent Computation

Creating Domain Specific Languages in F#

F# Type Providers in Depth

Asynchronous programming in F# (QCon 2012)

Queries in general purpose languages

Docase notation for Haskell

Accessing loosely structured data from F# and C#

F# on the Server-Side

F# Tutorial @ QCon

Teaching F#

F# in MonoDevelop

Academia

Concurrent programming with Agents

Recently uploaded

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

Scaling API-first – The story of a global engineering organization

Radu Cotescu

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

Real Time Object Detection Using Open CV

Khem

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Recently uploaded (20)