Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sq lite module1

26 views

Published on

Module 1 out of 9; SQLite training slides, databases, SQL, ERD, software design, database

Published in: Software
  • Be the first to comment

  • Be the first to like this

Sq lite module1

  1. 1. MODULE 1: SQL AND SQLITE INTRODUCTION SQL and Scripting Training (C) 2020-2021 Highervista, LLC 1
  2. 2. 2 TOPICS Course Introduction ­ About Ron McFarland Highervista LLC ­ About the Textbook used for this course Introduction to Databases ­ Proliferation of Data ­ Popularity of Databases ­ Data vs Information ­ Types of Databases ­ Importance of Database Design ­ Evolution of Modern Database Systems ­ System Data Management ­ Components of a Database System ­ Functions of a Database Management System (DBMS)
  3. 3. INTRODUCTION SQL and Scripting Training
  4. 4. 4 INTRODUCTION Ron McFarland Technologist, Educator Source: Microsoft Images
  5. 5. 5 ABOUT THIS COURSE This course is distributed free. I use several sources. But importantly, I use the book noted on the next slide. If you are using these PowerPoints, please attribute Highervista, LLC and me (Ron McFarland). IN ADDITION, please attribute the author noted on the next slide, as the author’s textbook provides essential information for this course. Source: Microsoft Images
  6. 6. 6 INTRODUCTION This course is offered to you free. HOWEVER, please purchase the following book, as it is a primary resource for this course. I do not make any $ from this course or this book. So, since a handful of good content is derived from the following text, please support this author! Title: SQL Quickstart Guide Author: Walter Shields Available: Amazon, B&N, and through ClydeBank media website at: https://www.clydebankmedia.com/books/programming- tech/sql-quickstart-guide
  7. 7. 7 CONNECT WITH RON MCFARLAND Ron’s website: http://www.highervista.com LinkedIn: https://www.linkedin.com/in/highervista/ Source: Microsoft Images
  8. 8. COURSE OVERVIEW SQL and Scripting Training
  9. 9. 9 COURSE DESCRIPTION Study of the functions, underlying concepts, and applications of Structured Query Language (SQL) and relational database management systems (RDBMS) in a business environment. The aim is to appropriately use SQL and the RDBMS to meet business requirements. Discussion of entity/relationship diagrams (ERD), relational theory, normalization, integrity constraints, SQL, and physical and logical design. Additional advanced SQL topics include complex data types, functions, subqueries, and views.
  10. 10. LEARNING TOPICS AND PREREQUISITE SQL and Scripting Training
  11. 11. 11 LEARNING TOPICS •An introduction to databases used in the organization •Database types •The Relational Database model •How to design a relational database •How to design and create your own database •Use of SQL for relational databases •SQLite as the primary database "engine" •Using a sample database (sTunes) with the SQLite database engine •Using a database browser for SQLite •Implementing a database design in SQLite •Scripting commands that support SQLite •How SQLite works with other programming languages oPython
  12. 12. 12 COURSE PREREQUISITE Preferred Competency and experience (academic or job-related) in: ­ Finding solutions to problems using structured programming and step-wise refinement techniques with a programming language.* ­ Use of pseudocode or other design techniques for software development. ­ Hands-on practice in software debugging, testing, and documentation. ­ Use of algorithms to solve problems.
  13. 13. 13 REQUIRED RESOURCES SQLite Home Page: https://sqlite.org/index.html SQLite Browser: https://sqlitebrowser.org/ Draw.io: type in draw.io in your browser (for the ERD). Various YouTube and Resource links. Use of a sample SQLite database (provided). Word processor for Project Documentation. Spreadsheet for Data Dictionary
  14. 14. 14 REQUIRED RESOURCES Hardware Resources for SQLite database engine: PC with Windows 10 Mac with Catalina 10.14 or higher ­ Note: SQLite can be used on Android, MacOS, Linux, etc. Browsers for other tools in the course: ­ Common browsers include Chrome, Firefox, Safari (Mac). ­ DB Browser for SQLite (a GUI for the SQLite databases) ­ Draw.io (for data models) McFarland (2020)
  15. 15. INTRODUCTION TO DATABASES SQL and Scripting Training
  16. 16. 16 PROLIFERATION OF DATA Proliferation of data throughout organizations and the web ­ Customer Data ­ Document & Document Management Systems ­ Communications & Communication Systems ­ Backups & Backup Storage ­ Transactional Data ­ Social Data ­ Sensors & Machines (IoT, SCADA, etc.) Source: Spacey, J. (2016, February 4). Source: Microsoft PowerPoint images
  17. 17. 17 DATA USAGE Source: Clement (n.d.). Robust volume of internet data in the United States
  18. 18. 18 INTERNET PENETRATION RATES Source: Clement (2020, February 14) Countries with the highest internet penetration rates as of 2020
  19. 19. 19 TYPES OF INTERNET USERS: US Source: Clement, J. (2020, February 18)
  20. 20. 20 DATA OVER THE INTERNET Google 2000—14 billion searches per year. 2020—7 to 10 billion searches per day. YouTube 2017—Over 400 hours of video content uploaded per minute. 2019—500 hours per minute. Facebook, Instagram, WhatsApp, Messenger 3.14 billion active users each month.
  21. 21. 21 EXPONENTIAL GROWTH Source: Smith (2020, July 11)
  22. 22. 22 IMPACT Every Google search, Facebook post, YouTube video request, and Amazon product review is stored and managed on a server or servers subject to query. SQL is the principal tool to decipher and cull information from a multitude of data sources. As a result, SQL skills are in demand.
  23. 23. 23 INTERESTING FACTS ABOUT DATA Only ½ of 1 percent of all data collected is analyzed. Effective utilization can improve existing markets, open new markets, support more effective fraud prevention, and yield other benefits. What impacts do you envision could occur if a higher volume of data is analyzed?
  24. 24. 24 DATABASES FULFILL NEEDS Scientific and business need to group data, determine its inter- relationships and come to conclusions. Organizational need to determine data quantity, quality, place, time, position and action. Source: Microsoft PowerPoint Images
  25. 25. 25 WHAT DATABASE MANAGEMENT CAN DO 1. A Database Management System Is an Extension of Human Logic 2. Computational Power: Computers Can Quickly Answer Lots of Questions 3. Solve Complex Problems: Some Questions Can Be Really Complicated 4. Handle Large Volumes of Data: We Are Easily Overwhelmed With Information 5. More Efficient Than Humans: Automation Is the Key to Efficiency 6. Better Than a Manual Process: A DBMS Is Better Than Manual Processes in so Many Ways 7. Save Money!
  26. 26. 26
  27. 27. 27 DATA VS INFORMATION Data are facts. The number 1,005 is data. Without analysis and processing, facts do not have much value in business decision making. Data—raw facts. Information—processed, refined, and aggregated data, or data in a context. October 5 (a date) vs October 5 (my birthday). Information and knowledge support decision making. Source: Microsoft Images (2020)
  28. 28. 28 DATA OR INFORMATION?
  29. 29. 29 DATA OR INFORMATION? Data are basic facts Information has context Report shows what student is enrolled in a course on a given date. These facts have little value out of context. McFarland (2020)
  30. 30. 30 DATA OR INFORMATION? This report shows how many total students enrolled in each course and course popularity based on enrollment. It can be used for business decisions such as adding sections of popular courses.
  31. 31. 31 TRANSFORMING DATA INTO INFORMATION When data is stored, it can be used as input for an information system. An information system has programs to process (or transform) data to produce information as an output. Information reveals the meaning of data.
  32. 32. 32 TRANSFORMING DATA INTO INFORMATION Example: Student data values such as ID, Name, Address, Major, and Phone Number represent raw facts. Class roll is a list showing students IDs and Names of those enrolled in specific class (course section).
  33. 33. 33 VIDEO: DATA-INFORMATION-KNOWLEDGE IN 3 MINUTES Link:https://www.youtube.com/watch?v=sIjSY05JE9Q&feature=emb_logo
  34. 34. 34 TYPES OF DATABASES: WHAT IS A DATABASE? An electronic filing cabinet Contains persistent storage Users can insert new info, or change, retrieve, or delete CRUD: Create, Read, Update, Delete
  35. 35. 35 TYPES OF DATABASES: PURPOSE A database has a purpose, or focus
  36. 36. 36 TYPES OF DATABASES: LOGICAL EXAMPLE • Table S: Suppliers under contract • Table P: Parts • Table SP: Shipments
  37. 37. 37 LOGICAL VS PHYSICAL DATABASE
  38. 38. 38 LOGICAL VS PHYSICAL DATABASE What the user sees. User can be end user or developer. (logical design) How the DBMS stores the data (physical Implementation)
  39. 39. 39 DATABASE MANAGEMENT SYSTEM (DBMS) Note: Previous slide showed database from the user’s perspective (logical database) What distinguishes a database from a Database Management System (DBMS)?
  40. 40. 40 DATABASE MANAGEMENT SYSTEM (DBMS) Essential Definition of a DBMS: Software designed to store, retrieve, define, and manage data in a database.
  41. 41. 41 KEY DIFFERENCES BETWEEN A DBMS AND RDBMS
  42. 42. 42 WHAT IS A DBMS?
  43. 43. 43 TYPES OF DATABASES Data must be stored and organized properly to be processed easily, so that accurate information is generated. Databases are created to fulfill the task—the proper generation, storage, and retrieval of data. Even though they all share some common characteristics and benefits such as data independence, data sharing, transaction control, backup and recovery, there are various types of databases.
  44. 44. 44 TYPES OF DATABASES Based on number of users ­Single-user database ­ Supports only one user at a time. ­ Examples: Microsoft Access, Microsoft Excel, SQLite, file-based data storage. ­Multi-user database ­ Supports multiple users simultaneously. ­ Examples: Oracle, MySQL, MS SQL Server.
  45. 45. 45 TYPES OF DATABASES Based on database location(s) ­Centralized database—data located at a single site. ­Distributed database—data distributed across multiple sites. Based on data usage ­Operational database—supports day-to-day operations. ­Data warehouse—data aggregated from operational databases to support strategic business decision making.
  46. 46. 46
  47. 47. 47 COMMON DATABASE CATEGORIES
  48. 48. 48 ROLE OF THE DBMS The DBMS manages the interaction between end users and the database.
  49. 49. 49 KEY DATABASE TERMINOLOGY: METADATA Data about data. Defines data characteristics such as the data type (character or numeric) and the relationships that link the data.
  50. 50. 50 KEY DATABASE TERMINOLOGY: RELATIONSHIPS An important component of database design. Often defined by their environment.
  51. 51. 51 KEY DATABASE TERMINOLOGY: RELATIONSHIPS Example: Employee and Job The relationship between EMPLOYEE and JOB is likely to depend on the organization's definition of the work environment. In some organizations an employee can have multiple job assignments, while in other organizations—or even in other divisions within the same organization—an employee can have only one job assignment.
  52. 52. 52 DATA MODELS The details of relationship types and the roles played by those relationships are portrayed in data models. Source: Microsoft Images
  53. 53. 53 TRANSACTION (PRODUCTION) DATABASE Reflects the daily operations of an organization. Records transactions such as the sale of a product, the enrollment of a student, or the opening of a checking account. Transactions are time-critical and must be recorded accurately and immediately.
  54. 54. 54 DATA WAREHOUSE Data warehouse databases focus primarily on the storage of data to be used to generate information for tactical and strategic decision making. Such decision making typically requires extensive data manipulation to extract information from data. In short, the data warehouse data are the source for decision support systems. Data warehouse databases are far less time-critical than transaction databases; the former record the historical data record that will be used to formulate pricing decisions, sales forecasts, market positioning, and so on. Data warehouse databases typically store complex data derived from many sources.
  55. 55. 55 IMPORTANCE OF DATABASE DESIGN A well-designed database will Facilitate effective data management. Generate accurate and valuable information to assist business decision making. A poorly designed database Will not achieve goals. Can lead to bad decision making and organizational failure.
  56. 56. 56 DATABASE DESIGN: CAVEATS Good application programs can't overcome bad database designs. The existence of a DBMS does not guarantee good data management, nor does it ensure that the database will be able to generate correct and timely information. Ultimately, the end user and the designer decide what data will be stored in the database.
  57. 57. 57 DATA MODEL: THE BLUEPRINT The data model to database creation is what a blueprint is to home building. A database created without the benefit of a detailed blueprint (data model) is unlikely to be satisfactory. Source: Microsoft Images
  58. 58. 58 DATABASE DESIGN: TIPS TO REMEMBER Good database design avoids redundant data (unnecessarily duplicated data). A poorly designed database generates inconsistent data (inaccurate data). This can lead to bad decisions and even failure of the organization.Source: Microsoft Images
  59. 59. 59 EVOLUTION OF THE MODERN DATABASE: MANUAL FILE SYSTEM Traditionally composed of collection of file folders kept in file cabinet. Organization within folders was based on data's expected use (ideally logically related). System was adequate for small amounts of data with few reporting requirements. Finding and using data in growing collections of file folders became time consuming and cumbersome. Source: Microsoft Images
  60. 60. 60 CONVERSION FROM MANUAL FILE SYSTEM TO COMPUTER FILE SYSTEM Could be technically complex, requiring hiring of data processing (DP) specialists. DP specialists created file structures, wrote software, and designed application programs (manually designed databases that were not consistent). Resulted in numerous "home-grown" systems being created. Initially, computer files were similar in design to manual files Source: Microsoft Images
  61. 61. 61 INITIAL CONVERSION OF MFS TO COMPUTER FILES Source: DBMS Components (2020)
  62. 62. 62 TERMINOLOGY OF COMPUTER FILE SYSTEMS Source: DBMS Components (2020)
  63. 63. 63 DEPLOYMENT OF A SIMPLE FILE SYSTEM Source: DBMS Components (2020)
  64. 64. 64 CONTEXT: HISTORICAL BACKGROUND
  65. 65. 65 FROM FILE SYSTEM TO DATABASE DEVELOPMENT Data storage based on file system had shortfalls that could lead to inaccurate, outdated, or inconsistent information. That led to some innovations in database systems. Modern, integrated database approach to implementing data storage has many advantages over small, isolated databases with "islands of information" Source: Microsoft Images
  66. 66. 66 EVOLUTION OF THE MODERN DATABASE Manual file systems (MFS): Punch cards Computerized file system: Paper tape, magnetic tape, magnetic disk, and newer devices. Direct File Processing: Direct access devices such as disks facilitated replacement of sequential file processing (ISAM, VSAM).
  67. 67. 67 Systemwide Data Management Process
  68. 68. 68 INTEGRATED DATABASES Facilitates sharing of data so that many users can access the same piece of information. Can reduce redundancy and minimize data inconsistency as multiple copies of the same data are not stored unless necessary. Improves data security as an enterprise database allows security restrictions to be defined and enforced on several levels. Can improve data integrity, better balance conflicting requirements, develop new applications faster, and provide better data accessibility, backup, and recovery procedures.
  69. 69. 69 DATABASE SYSTEM COMPONENTS Hardware: Physical devices such as computers, I/O channels, storage devices, and network devices. Software: Operation system software, networking software, DBMS software, application programs and utility software. Data: The most important component. Collection of data (business operational facts), as well as metadata (data about data or description about data), stored in the database.
  70. 70. 70 DATABASE SYSTEM COMPONENTS Procedures: Instructions, processes, specifications and rules that govern the design and use of the database system. People: Users and stakeholders. Database designer Developers System analysts Application programmers Database administrators End users
  71. 71. 71 THE DBMS
  72. 72. 72 DBMS: IMPORTANT FUNCTIONS Data dictionary management: stores and manages metadata, or definition of data elements and their relationships, in the database. Data storage management: creates and manages complex structures required for data storage. Data transformation and presentation: converts entered data based on pre- defined format and standard; retrieves and presents data to end users in proper, user-friendly format. Security management: safeguards data as well as the complete system from unauthorized access; enforces data integrity and data privacy. Multiuser access control: uses sophisticated algorithms to ensure that multiple users can access the database concurrently without compromising the integrity and consistency of the database. Source: Microsoft Images
  73. 73. 73 DBMS: IMPORTANT FUNCTIONS Backup and recovery management provides special utilities for routine and special backup and restore procedures; manages the recovery of the database after a failure, such as a bad sector in the disk or a power outage. Data integrity management implements and enforces integrity rules to minimize data redundancy and maximize data consistency. Source: Microsoft Images
  74. 74. 74 DBMS: IMPORTANT FUNCTIONS Database access languages and application programming interfaces provides data access through a standard query language, or SQL (Structured Query Language). This non-procedural language allows users to specify what must be done without having to specify how. Database communication interfaces serves users from multiple, different network environments. Source: Microsoft Images
  75. 75. 75 DATABASE MGMT SYSTEM: 10 GENERAL FUNCTIONS To ensure data integrity and consistency, a database has 10 general functions. Functions common to systems including ­ hierarchical ­ network ­ relational ­ big data ­ object oriented
  76. 76. 76 1. FUNCTIONS: DATA DICTIONARY MANAGEMENT Data Dictionary—where the DBMS stores definitions of the data elements and their relationships (metadata). Used by DBMS to look up the required data component structures and relationships. When programs access data in a database they are basically going through the DBMS. McFarland (2020)
  77. 77. 77 2. FUNCTIONS: DATA STORAGE MANAGEMENT Used to store data and related data entry forms or screen definitions, report definitions, data validation rules, procedural code, and structures that can handle video and picture formats. Users do not need to know how data is stored or manipulated.
  78. 78. 78 2. FUNCTIONS: DATA STORAGE MANAGEMENT
  79. 79. 79 3. FUNCTIONS: DATA TRANSFORMATION AND PRESENTATION Exists to transform data entered into required data structures. Using the data transformation and presentation function, the DBMS can determine the difference between logical and physical data formats.
  80. 80. 80 4. FUNCTIONS: SECURITY MANAGEMENT Security management sets rules that determine specific users allowed to access the database. Example: Username and password or biometric authentication (such as a fingerprint or retina scan). The latter are more costly. The security management function also sets restraints on the specific data any user can see or manage.
  81. 81. 81 5. FUNCTIONS: MULTIUSER ACCESS CONTROL Data integrity and data consistency Enables multiple users to access the database simultaneously without affecting database integrity.
  82. 82. 82 5 ABAC AND RBAC
  83. 83. 83 6. FUNCTIONS: BACKUP AND RECOVERY MANAGEMENT Backup and recovery a consideration for dealing with outside threats to a database. Example: How long does it take to recover the database after a power outage? Backup management ­ data safety and integrity ­ example: backing up your mp3 files on a disk Source: Microsoft Images
  84. 84. 84 7. DATA INTEGRITY MANAGEMENT The DBMS enforces rules to reduce data redundancy, maximize data consistency, and ensuring database consistently returns the correct answer for the same query.
  85. 85. 85 8. FUNCTIONS: DATABASE ACCESS LANGUAGES AND APPLICATION PROGRAMMING INTERFACES (API) How a DBMS can accept different end user requests through different network environments. Example: A DBMS can provide access to the database using the internet through many web browsers (Mozilla Firefox, Internet Explorer, Netscape), via the API. A language environment, like the C programming language or the Python programming language, can interface with a database, via the API.
  86. 86. 86 8. FUNCTIONS: DATABASE ACCESS LANGUAGES AND APPLICATION PROGRAMMING INTERFACES
  87. 87. 87 9: FUNCTIONS: DATABASE COMMUNICATION INTERFACES How a DBMS can accept different end-user requests through different network environments. An example of this can be easily related to the internet. A DBMS can provide access to the database using the Internet through Web Browsers (Mozilla Firefox, Internet Explorer, Netscape).
  88. 88. 88 10. FUNCTIONS: TRANSACTION MANAGEMENT A DBMS must supply a method that will guarantee that all the updates in a transaction are made or not made. ACID (Atomicity, Consistency, Isolation, and Durability)
  89. 89. 89 REFERENCES Clement, J. (2020, February 14). Countries with the highest internet penetration rate 2020. Retrieved August 28, 2020, from https://www.statista.com/statistics/227082/countries-with-the-highest-internet-penetration-rate/ Clement, J. (2020, February 18). United States digital population 2020. Retrieved August 28, 2020, from https://www.statista.com/statistics/1044012/usa-digital-platform-audience/ Clement, J. (n.d.). Topic: Internet usage in the United States. Retrieved August 28, 2020, from https://www.statista.com/topics/2237/internet-usage-in-the-united-states/ McFarland, R. (2020). Published Articles: Ron McFarland. Retrieved December 03, 2020, from https://medium.com/@highervista Smith, C. (2020, July 11). 150 Amazing Amazon Statistics. Retrieved August 28, 2020, from https://expandedramblings.com/index.php/amazon-statistics/ Spacey, J. (2016, February 4). 7 Examples of Data Proliferation. Retrieved August 28, 2020, from https://simplicable.com/new/data-proliferation
  90. 90. 90 LINKS Differences between data and information: https://blog.udemy.com/difference-between-data-and- information/ Database Characteristics and Benefits: https://opentextbc.ca/dbdesign01/chapter/chapter-3- characteristics-and-benefits-of-a-database/ DBMS Classification: https://opentextbc.ca/dbdesign01/chapter/chapter-6-classification-of- database-systems/ DBMS Components (2020): http://computer-dbms.blogspot.com/2011/09/components-of-dbms.html DBMS Functions: http://www.myreadingroom.co.in/notes-and-studymaterial/65-dbms/465-functions- of-dbms.html Advantages and Disadvantages of DBMS: http://www.myreadingroom.co.in/notes-and- studymaterial/65-dbms/462-advantages-and-disadvantages-of-dbms.html

×