How to start making complex software?

By complex software I mean large programming projects, lots of files, lots of code. How should I start, what technical aspects do I need to know? I've heard that to make a big and well programmed software, you should know thoroughly the programming language you use, the data structure and algorithms, the optimization tools (like Big-O notation, for example) and the software architecture (probably, there are more aspects to consider). I've made a simple 2D game engine for Windows before, and it works pretty well, but the source code? It's pretty messy code. Adding new things or modifying things was difficult. The system in general was hard to understand. I want to create a well-structured, efficient, and well-programmed system. To solve this problem, I plan to read Clean Architecture: A Craftsman'S Guide to Software Structure and Design (Robert c. Martin). am I doing it right? Any suggestions or personal experiences on this topic are welcome, I'd love to read them.

>>40183 >step 1 Identify your requirements >step 2 ??? >step 3 Profit

>>40183 >A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. https://en.wikipedia.org/wiki/John_Gall_(author)#Gall's_law

>>40183 I don't have a solid process, but for large solo projects I always start with sketching out the idea on paper, starting from a high level and then focusing on smaller components. If I feel I've gone into enough detail and/or I'm not sure an idea will work well I will write code for some components until I find a problem or ambiguity with my (sketched out) idea. Then I go back to sketching, repeat.

>Step 1: make simple software >??? >Step ?: congrats, you have complex software

>>40183 massive codebases are almost never good >lots of files, lots of code this should be lots of libraries, which then get used by "bigger" projects for example, in a project of mine I use nested sets to implement a permissions system. rather than creating a file for the nested sets in the project, I implemented it as a library which is then imported by the project at an old job of mine I saw many codebases where generic functionality ends up getting implemented in non-ideal locations. this results in tech debt and reimplementation of ideas across projects

>Clean Architecture: A Craftsman'S Guide to Software Structure and Design (Robert c. Martin) jesus christ please dont read anything from this guy, it will poison your mind, anything from uncle bob is a meme

Keep your mind on a few key concepts Every class (whether "in-code" or effective) gets its own file. Any time when you're dealing with state, you want a single source of truth. Any time you're dealing with other people's code, you want excessive logging to show where they fuarrrked up. Any time you're dealing with a user's active input, you're a mirror: Model, View, Controller is Brain, Eyes, Hands in reverse. Any time you're dealing with a user's input, you need to put nay task that takes more than 80ms or relies on network activity in its own thread/process/worker. If the language requires you to color your functions async, just do it from the start. Every action is against you. Code against the heretic and the mutant error and the hostile action from the start. You're an engineer, not an arborist, you plan for the worst day, not the average one. You need to know at least two languages well: a very low-level (C/Rust/Zig/Dart) one and a very perfect one (Python). I chose C and Python, and no, JS does not count for either. You need to understand how a computer works and how a language works and have access to the benefits of both. If you allow Matlab code in prod, you'll regret it. Yes, yes, they can "compile it to C now", ropemax your neck. Learn how to use Eigen in C/C++, it's an easy way to get the benefits of Matlab-level math and all the math nerds will worship the ground you walk on. If you want mass adoption, make modification easy. People change things when they break, and don't change things that don't break. Don't break. Your boss, coworkers, peers, gf, alien civilizations, &c. interact with you as either a problem or as a solution. You don't need to be a solution, but don't be a problem.

Unironically get a job. There are general guidelines but in the end most of complex codebases started being small and grew. Good engineers can make these codebases grow in a somewhat orderly and extensible manners, learn from them. Shitty engineers just make a mess, that you can do on your own.

>>40418 I've been hearing that recently but I'm not sure why, care to expand on it?

>>40675 a way to put it is that uncle bob's books belong to a different era. in the late 90s and 00s OOP was the new hot thing, and the main "ideology" in software engineering was something that could be described as "object oriented fundamentalism", and that pushed hard for overabstraction and to make full use of the object oriented paradigm even if that meant turning 10 lines of code into 5 classes and 100 lines of boilerplate. we've since realized that it was deeply retarded and today you'd be crucified by your team if you wrote code the way old users like uncle bob did - and rightfully so. the modern idea of what good code should look like is much more about a synergy of FP and "soft" OOP, which again is very much unlike what uncle bob preaches.

>>40680 Are there good resources to learn about that modern method in a systematic way? I've only coded small projects and always get lost in large codebases.

>>40685 i would recommend you read pic related. it's a Scala book but in a way it doesnt matter, since it teaches the general principles behind FP that you can apply in more or less any modern language.

>>40686 Thanks, I tried my hand at LISP and Haskell in the past but dropped it because I didn't really have an use for it

>>40692 yeah, LISP and Haskell are a bit more on the academician and "hacker" side of things. Scala is the most "practical" among functional languages.