(untitled)

I have an idea for a "web 2.0" website I want to make but it involves a potentially huge database, and I have no experience with DBMS, or web 2.0 in general. does anyone have any good recommendations for guides that will point me in the right direction in terms of writing a fast, manageable database and setting up a website on a basic linux system? I already know how to program, write SQL, javascript, HTML, CSS, HTTP requests and I'll the things you'll imagine I need to know, so I'm not looking for a generic programming tutorial. just looking for recs, doesn't have to be a pdf download or anything.

>>3264 So you are asking for good recommendations for guides that will point you in the direction of writing a fast manageable database and setting up a website on a Linux host ? These are at least two separate things and you don't specify which things you are interested in ? To confirm , when you say writing a database, you don't mean writing your own DBMS from scratch , you mean designing a schema for data / information system using an existing database and designing optimized queries ? Web 2.0 is also a loaded term, and means different things to different people, see https://en.wikipedia.org/wiki/Web_2.0 for the one of the common accepted consensus definitions, and notice how it doesn't specify any particular technology, but the technology section mentions AJAX. Regarding pointing you in the right direction, here are my recommendations 1. Identify which server side programming language you are going to use for the backend of your web application, website's with huge databases and users with participatory content stored in databases are web applications and not just websites. Once you have identified this this will influence your other decisions. 2. Regarding DBMS choice. Choose an open source solution, because it will cost less than Oracle, IBM DB2 or Microsoft SQL Server. If you care about data integrity and performance then choose PostgreSQL over MySQL. I know both, but I recommend PostgreSQL for new projects. If you have a specific reason for using MySQL or MariaDB then use it, but be aware that the learning curves are similar. For other's people discussions on this see https://hackr.io/blog/postgresql-vs-mysql https://www.2ndquadrant.com/en/postgresql/postgresql-vs-mysql/ https://www.anchor.com.au/hosting/dedicated/mysql_vs_postgres https://di.nmfay.com/postgres-vs-mysql https://logz.io/blog/relational-database-comparison/ https://news.ycombinator.com/item?id=14070930 And your favourite search engine for additional discussions. 3. Regarding database design If you have no experience with information systems design or database design, you at least want to learn how to do that before starting. For example Resources you want to have read or use include - The first attached PDF of How not to structure your database-backed web applications: a study of performance bugs in the wild , also see https://blog.acolyer.org/2018/06/28/how-_not_-to-structure-your-database-backed-web-applications-a-study-of-performance-bugs-in-the-wild/ - https://use-the-index-luke.com/ (General advice on SQL Indexing and Tuning e-Book) - https://snook.ca/archives/building_a_web_application/building_a_web_1 - If you are going with PostgreSQL, then the second attached PDF which is PostgreSQL for Data Architects: Discover how to design, develop, and maintain your database application effectively with PostgreSQL - Other general database design texts which cover database and information systems modelling and design. See follow up posts for more examples Personally I prefer and recommend ORM2 / NIAM over ER diagrams and UML entity modelling, but you can use either. See https://en.wikipedia.org/wiki/Object-role_modeling and https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model for more information as well as the third attached PDF which is Information Modeling and Relational Databases by Terry Halpin and Tony Morgan. You want to learn about data normalization (Canonical Normal Form , 3rd Normal Form,etc) and then learn when it is a good idea to ignore it one some tables for performance reasons. You want to learn about primary keys and foreign keys and how they impact data integrity, data mass import and mass export (dataflow). For further recommendations around fast, manageable database and setting up a website on a basic linux system, see the follow up post to this.

>>3266 Other recommendations regarding database management systems include, once you have your back end language chosen, identify if you are going to write all SQL by hand ? write all SQL as stored procedures ? write SQL using mapped statements with something like ibatis or ibatis.net ? Use an ORM to abstract which database you are using (not the same ORM abbreviation as before, see https://en.wikipedia.org/wiki/Object-relational_mapping and https://en.wikipedia.org/wiki/Comparison_of_object-relational_mapping_software for more information ? Use an ORM to prevent you from writing SQL directly ?. This choice impacts your ability to maintain your web application and to ensure high performance when accessing database resources without using a cache. See attached resources for more information about high performance database configuration: 1. PostgreSQL Performance Tuning BRUCE MOMJIAN 2. PostgreSQL 10 High Performance: Expert techniques for query optimization, high availability, and efficient database maintenance by Ibrar Ahmed, Gregory Smith, Enrico Pirozzi 3. High Performance MySQL, 3rd Edition: Optimization, Backups, and Replication Also see https://wiki.postgresql.org/wiki/Performance_Optimization and https://www.mysql.com/why-mysql/performance/index.html https://dev.mysql.com/doc/refman/8.0/en/optimization.html You also want to learn about setting up a website on a basic linux system, then see the following resources You will also want to pick a web server, most people pick lighthttpd, Apache or nginx. If you haven't done web server administration before, then I recommend starting with nginx first. See first attached PDF in next post for more information on NGINX configuration specifically. Various Web Server Guides https://www.tecmint.com/linux-web-server-guides/

>>3269 Regarding Linux administration and web server administration, you will want to know Linux command line basics, and how to setup both the database and web server on the Linux host (whether cloud hosting, shared hosting, dedicated host, etc) If you are following my other recommendations, use Centos or Debian on your servers if you are new to Linux and use their stable versions. Find the additional attached resources 1. Complete NGINX Cookbook. Advanced Recipes for Operations by Derek DeJonghe 2. UNIX and Linux System Administration Handbook by Evi Nemeth and others 3. PostgreSQL Administration Cookbook by Simon Riggs; Gianni Ciolli; Gabriele Bartolini Alternatively see appropriate resources like https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/pdf/system_administrators_guide/Red_Hat_Enterprise_Linux-7-System_Administrators_Guide-en-US.pdf and https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/pdf/security_guide/Red_Hat_Enterprise_Linux-7-Security_Guide-en-US.pdf or other resources for Apache configuration etc.

>>3266 >>3269 >>3270 thank you for taking the time to type this all out, this is perfect. I planned on using Haskell as the backend language. and yeah I didn't mean from-scratch DBMS. >identify if you are going to write all SQL by hand ? as opposed to what?

>>3276 As opposed to the other choices of having an ORM or a query builder library generate parts or all of your SQL for you listed in my earlier post. See https://wiki.haskell.org/Web/Databases_and_Persistence for language specific examples. Specific advice regarding using Haskell as the back end language is below: See the list of Haskell web frameworks at https://wiki.haskell.org/Web/Frameworks and previous discussion on Haskell reddit https://www.reddit.com/r/haskell/comments/9vsaax/current_best_webframework_to_choose_in_haskell/ https://www.reddit.com/r/haskell/comments/8bfjy5/building_a_webapp_where_to_start_overwhelmed_by/ and an old comparison at http://edofic.com/posts/2014-02-23-haskell-web.html on picking a framework. I don't write much Haskell but I would recommend using a framework if I was going to write a web application in any langauge (Haskell, Python, PHP, C++, Java, etc), so I recommend using a framework here unless your solution is so minimal that you feel you can write your own libraries for everything. Regarding library choice with Haskell and databases , keep it simple to start with See http://hackage.haskell.org/package/postgresql-simple and http://hackage.haskell.org/package/mysql-simple as starting points. Regarding the third attachment , it is a bit dated now, and https://github.com/tssm/up-to-date-real-world-haskell is trying to update to reflect changes in the Haskell ecosystem. If you want an example of a Haskell based web application, then one whose source code I have perused is https://github.com/ahushh/Monaba . Please find attached the following resources. 1. Developing Web Apps with Haskell and Yesod: Safety-Driven Web Development [2 ed.] by Michael Snoyman. 2. Practical Web Development with Haskell: Master the Essential Skills to Build Fast and Scalable Web Applications [1 ed.] by Ecky Putrady 3. Real World Haskell by Bryan O'Sullivan, John Goerzen, Don Stewart