Python 3

Python 3 Welcome Lainons! (whether you are new to programming or more experienced with Python 3) Post your code and questions ITT. Old thread >>16053 >https://www.python.org (be sure to check out Library Reference and Tutorial) I recommend Python 3 for new programmers for these reasons: * It's popular. There are a lot packages/libraries and it's easy find help when you encounter problems. * Interpreted dynamic/high-level languages give you immediate feedback! * There are also a lot of jobs available for Python programmers, and there are a lot of FOSS projects to contribute to. Before you can write any code, you need to: 1. Install Python 3 (duh). On GNU/Linux, it's (almost) always installed by default (but remember to install tk and python-pip as well). 2. You need to get a text editor or IDE. Popular choices are: >Notepad++ or Kate >VSCode/VSCodium >NeoVim or GNU Emacs (Be sure to go through the tutorial that NeoVim and GNU Emacs have). >PyCharm 3. (optional but highly recommended) Install pylint (the package is probably called "python-pylint" or similar) and use it to check your code. Some books/resources: Beginner >Be sure to learn Python 3 (Python 2 is dead! And as a result, some tutorials/books are obsolete!). * Think Python 2nd edition: https://greenteapress.com/wp/think-python-2e/ (gratis. suitable for the absolute beginner) * Automate the Boring Stuff with Python: https://automatetheboringstuff.com (gratis. more practical) * Learning Python by Mark Lutz (warning: this book is huge!) dl it here: >>16679 * Learn Python the Hard Way by Zed Shaw * Try finding a MOOC (for example here: https://www.mooc.fi/en/#courses) Intermediate/Advanced * Fluent Python by Luciano Ramalho * Programming Python by Mark Lutz * Python Cookbook by David Beazley and Brian K. Jones * Effective Computation in Physics by Anthony Scopatz and Kathryn D. Huff (but be sure to read the TOC, it seems like a bit weird book) What can Python be used to do? A lot of things. Examples: * System maintenance, automation and Web scraping (check out Automate the Boring Stuff with Python (and the libs it uses). Also: Scrapy) * GUIs (it comes with TKinter but you can also use PyQT) * Games (PyGame) and Visual novels (RenPy, Katawa Shoujo was made using this.) * Pictures (Pillow) * Math, scientific computing & data-science (Matplotlib, NumPy, SciPy and Pandas. See also: Jupyter) * AI, Deep learning/machine learning, etc. (PyTorch, TensorFlow, Keras and OpenCV-python) * Language and linguistics (NLTK, Stanza, spaCy) * Web programming (Django and Flask) Tips for all Python programmers * Hint: you can upload .txt files or you can use a pastebin service (like https://dpaste.com) * Using GNU/Linux distro (like Linux Mint) is recommended. Use Rufus to create bootable flash drive (or you can use VirtualBox). You can use Windows disk management tools to shrink a partition to reclaim free space (I recommend ~30GB). * Use version control like Git (https://www.atlassian.com/git/tutorials) or Fossil because keeping track of copies/backups is hard and prone to errors (you can lose your work). Out of these 2, Fossil is probably easier to use. * Using venv or virtualenv is recommended when you install Pypi/Pip packages, See: https://opensource.com/article/19/4/managing-python-packages * Try running import this in interactive Python console. * Style guide: https://peps.python.org/pep-0008/ * If you gotta go fast: https://www.pypy.org * You can find exercises at these sites: https://exercism.org ; https://adventofcode.com ; https://projecteuler.net ; https://leetcode.com/problemset/all/ * Code examples: https://rosettacode.org/wiki/Python I compiled all of this because the old thread is full and I want to learn more Python (I don't currently understand OOP very well). Also, I have some questions: How to do unit testing properly and what to test? And the WTF is doctest?

>>33751 I am getting back into programming. I am working through Composing Programs (composingprograms.com). While I understand the material I think there aren't many exercises or practice problems like in SICP (which I never finished...). Could someone suggest some resources to give me stuff to work on?

>>33854 I looked into it and composing programs does have some amount of additional exercises. However, the course materials and resources are a far cry from SICP. I think I'll just go back to that and finish composing programs afterwards.

Just... Solve SICP problems with Python, the language doesn't really matter.

>>33877 This is a SICP version in Python: http://composingprograms.com/

I have a shell script that I run in a `subprocess.popen()` and would like to capture the output of that script and send it line by line into a logger object. The issue is that the output from the sub process is captured by the parentheses `()`, and I never see the output. Has anyone here ever seen this happen and would you be able to guide me on how to best do this? The bash code ```bash ( \ eval $(ssh-agent) > /dev/null; \ ssh-add -q /home/borgs/.ssh/borg-client_key; \ echo $borgPassword | \ ssh -p ${port} -A -o StrictHostKeyChecking=no ${user}@${ipAddress} \ "BORG_PASSPHRASE=\$(cat) \ borg \ --rsh 'ssh -p 9270 -o StrictHostKeyChecking=no' \ create \ --verbose \ --filter AME \ --list \ --stats \ --show-rc \ --compression lz4 \ --exclude-caches \ ssh://borgs@${ip_address}/~/${hostname}::$hostname-$backupTime $locations"; kill "${SSH_AGENT_PID}"; \ ) >&1 ``` The python code ```py def takeBackup(server, lg): print(f"Taking backup of {server['hostname']}") locationString = " ".join(server['locations']) print( "Running create-backup.sh command: create-backup.sh", f"{server['hostname']} {server['ip-address']} '{locationString}'" ) process = subprocess.Popen( [ "./create-backup.sh", server['hostname'], server['ip-address'], server['borgPassword'], locationString ], stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE ) for line in process.stdout: line = line.decode().rstrip('\n') lg.print_info(line) process.wait() print(process.returncode) ```

Rewriting a C# game engine in Python. I noticed I enjoy optimizing stuff more than actually using my engine. >>33751 > How to do unit testing properly and what to test? And the WTF is doctest? I learned about automated testing with "Obey the Testing Goat" by Harry Percival. The idea of automated tests is to assert a component of the software you wrote is working as intended. Notice the "you" and the "component". Don't test other people's code, and don't test several components at once (unless you are testing the integration of these components). A component should be an atomic piece of software that, given an input, return a known output. This is why easier to test functions than classes. If you want test the behavior of objects who manage state you will have to learn to design for testability, to isolate the functional core of a program, and mocks. I have noticed that software that is easy to test is also easy to read, maintain, and document. You can start automating manual testing with functional tests. Try to reproduce the steps you usually take to test your code by hand using code. >doctest I have never used these buy they look good on self-contained utility functions.

How long would it take an average person with rock-solid Python knowledge to learn Django? Not that I intend to do it, but I'm wondering if it's really relevant for a company to select based on mastery of that kind of framework everyone knows about.

I've done coding tutorials online before, but have trouble doing things like working with visual studios, or installing linux, or once I install something like Python, getting it to do something interesting. I understand C++ and Python well enough, but don't feel at home working on the computer or getting it to do anything interesting. Are there good ways to get comfortable with this environment, or is it just practice?

>>34188 >visual studios You don't have to use visual studio or VS{Code,Codium} if you don't want to. You can start with Notepad++/Geany/Kate. I recommend you don't use full Visual Studio, unless you do C# development. Even then, VS{Code,Codium} has most features that you need and full Visual Studio is very bloated and slow (even if you have a decent CPU and fast DDR4 RAM). Make sure to disable the telemetry if you use VSCode. >Are there good ways to get comfortable with this environment, or is it just practice? You need to test various tools (like text editors) and distros to find the right tools for you and a distro that you like. >or installing linux Download a installation iso and install VirtualBox and set up a VM. You should avoid bad distros (Manjaro, read below) or too obscure distros (like GoboLinux). To learn to use CLI/Bash, work through this book: https://linuxcommand.org/tlcl.php When you get into trouble, use ArchWiki: https://wiki.archlinux.org/title/General_recommendations But you should also consult the documentation of your distro. You can get real-time help on IRC (I recommend Hexchat IRC client. Most FOSS projects are on libera.chat network. Use a pastebin (like https://dpaste.com) to post log files and config files.). I recommend you pick a distro from this list: >Xubuntu or Linux Mint (these 2 are the easiest to use for a beginner) >Debian (https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/current/amd64/iso-cd/) >Devuan (it's Debian without systemd. I recommend you install OpenRC init when the installer asks) >Void Linux (glibc version if you are beginner because some programs don't support musl) >Arch Linux or Artix Linux (I recommend OpenRC init) But not Manjaro: https://manjarno.snorlax.sh >Gentoo NixOS and GNU Guix are also interesting but they are very different from a normal distro which is why I can't recommend them for a beginner. Arch/Artix, Void and Gentoo are made for power users, so you are expected to know how to use CLI/Bash and you need to be able to follow instructions from wiki pages/handbook. I highly recommend you start with Xubuntu or Linux Mint (mate version) but you can play around safely in a VM. Most GNU/Linux distros use systemd but it has caused some controversy (and there are valid reasons to not use it). But systemd vs other init systems is not *that* important when you are a beginner. But if you want to read more, here are some pointers: https://without-systemd.org/wiki/index_php/Arguments_against_systemd/ & https://nosystemd.org If you want to use a distro that doesn't have systemd, I recommend Void Linux (Glibc version). Use ext4 file system for Linux if the installer asks. You don't need swap partition, unless you want to suspend on disk (DragonFly BSD even deprecated swap! However, there are some counter arguments: https://chrisdown.name/2018/01/02/in-defence-of-swap.html Also: Never put your swap partition/swap file on a SSD.). Quick Win/Linux dual-boot setup HowTo * First, use Windows disk management tools to shrink a partition: https://www.diskpart.com/disk-management/shrink-volume-6289.html * Download installation iso. You can use PowersHell to check the integrity of the iso: Get-FileHash -Algorithm SHA256 gentoo.iso * Then use Rufus (https://rufus.ie/en/) to create an installation USB. If you get errors, I recommend you use the "dd" mode in Rufus. * After Rufus finishes, reboot your computer and use BIOS/UEFI settings/boot menu to select the USB flash drive. * You should be now booted into GNU/Linux live environment (you can test the distro and/or you can install it on your disk. Make sure to test that wifi works). * If you want to partition your disk using a GUI tool, you can use SystemRescueCD (https://www.system-rescue.org/) and use gparted for partitioning the disk.

How do I change the cipher suites used by Python?

I'm not a programmer, but I have messed around with shell scripts for fun and automation before. What are some cool things that can be done with Python that I can't achieve with a simple shell script and coreutils?

>>34399 >What are some cool things that can be done with Python that I can't achieve with a simple shell script and coreutils? If you do only quick and basic tasks, Bash and Python are comparable (and the choice doesn't matter much) but if you need to make something non-trivial, regular Bash/POSIX sh scripts usually become harder to understand. Python (and other proper programming languages) usually result in cleaner code. Sh relies heavily on external programs while Python comes with a huge stdlib (but you can also call external programs using subprocess.run). Also, Bash has more edge cases too: https://mywiki.wooledge.org/BashPitfalls It's also much easier to make a GUI in Python (you can find TKinter from the stdlib). Also, debugging Python script is easier vs shell script (and Python has better errors). Finally, while I haven't looked at any benchmarks, I'm fairly sure that Python is faster than Bash.

>>33751 >import this >... >There should be one-- and preferably only one --obvious way to do it. >... Then why have (bolted on) OOP at all? Isn't imperative paradigm enough? Why have both methods and functions (procedures)? Why didn't Python learn from Perl that bolted on OOP is not that good or useful for mainly scripting language. Food for thought: https://leontrolski.github.io/mostly-pointless.html https://news.ycombinator.com/item?id=25933121

>>36583 There is nothing bolted on about the OOP in python. Your critique is not coherent.

it is like kicking dead whales down the beach that the stdlib provides both functions and methods. For example, why you use list.remove and list.append but len(list)?

>>33751 > old thread Unless the thread got deleted you should nog do that make-and-discard “general” soykaf on low quality boards. The whole point of a slowboard like lainchan is that you can revive the old thread and have it going for years. Don’t pollute the catalog and destroy precious old threads by spamming the same “general” over and over again. fuarrrk off

>>36583 Python's OO is most definitely not bolted on. Python's type system is purely object oriented, all the way down to having metaclasses. But you're right in that OO style code is redundant with procedural style code.

>>33751 Python 3.11 and 3.13 introduced many breaking changes. I really don't want to deal with python after this point. Too many APIs were removed without sound justification. What did msilib do wrong? It's been shipped along with python since something like 2006.

>>36608 >https://discuss.python.org/t/pep-594-take-2-removing-dead-batteries-from-the-standard-library/13508 >Microsoft is slowly moving away from MSI in favor of Windows 10 Apps (AppX) as new deployment model It was probably pending for removal for several years.

>>36611 So what? It's not like we don't have old systems laying around. It's not the job of the Python project to adjust project timelines for microshaft. Absolutely insufferable language these days.

Hi lainons, I am trying to get a job of python but i have no profesional experience neither a degree so I am trying to create projects (flask and fastapi) but all my projects are so small i feel that i will never get a job. Any advice about projects to do?

>>36615 >t's not the job of the Python project to adjust project timelines for microshaft. But it's the job of the core devs to maintain all the modules included in the stdlib. I can't blame them for removing modules that almost nobody uses. > Absolutely insufferable language these days. It will only get worse. People are talking about Python 4, removing the GIL, and it seems everyone and their sister wants to add new "features" to the language every week. I wish they would focus on removing bugs and improving performance.

>>36623 >I wish they would focus on removing bugs and improving performance. I haven't run into a CPython bug since 2015. There have already been major performance improvements and they're working on more. Anyone who follows python would know about this stuff. Why don't you?

>>33751 I've begun studying Python and I read 5 chapters of this book "Think Python How to Think Like a Computer Scientist" but I found myself getting distracted a lot so I decided to follow the advice of this vid: https://youtu.be/QMbx0dTWJIQ?si=eYw4-s3yptYyELB1 I've done a 7 "projects" in total and I can say I've learned a lot more than I did reading. Although my concern is that I don't know the proper terminology for programming and that my knowledge of Python might remain surface level. So long story short should I continue with the book or can I continue with just pure coding?

>>36985 continue with the book, unless you know how to program in another language. there are some theoretical concepts that you need to know. And you really can't learn OOP just by doing (you need to read the theory, too).

>>33751 I'm working through the Python mooc (link in OP). If Python is so great, why doesn't it detect that "isupper" is a typo (missing pair of parentheses). # no errors detected in the code but it doesn't work # you need to manually see that it should be "itm.isupper()" instead. def rem_uppcase(lst : list) -> list: new_lst = [] for itm in lst: if(itm.isupper == False): new_lst.append(itm) return new_lst if __name__ == "__main__": testi = ["first", "SECOND"] for itm in rem_uppcase(testi): print(itm)

>>37177 Because it's not a typo, it's a valid Python program; you might want to compare functions instead of the result of their evaluation. Use mypy if you want static typing in Python.

>>37177 Python is a poor language, but not for any issues you've described. Python sort of encourages these low level loops which are hard to reason about. In this particular case, a lot of python programmers would break this into a comprehension. However for a large swaths of code, python does encourage a poor style like you've displayed above. For the code example instead of using the more pythonic list comprehension, I will use filter def rem_upper(xs): return list(filter(lambda x: not(x.isupper()), xs)) Note that function value of x.isupper is sometimes useful. I was tempted to use it as the f in filter, however the dot syntax and needing to compose it with not stopped using it as such.

>>37177 When you write your first interpreter you will know of why this kind of error can't be detected by a machine.

>>37177 In CL, by the way: (defun string-upcase-p (str) (declare (type string str)) (every #'upper-case-p str)) (defun rem-upcase (strlist) (remove-if #'string-upcase-p strlist)) (dolist (el (rem-upcase '("first" "SECOND"))) (format t "~A~%" el)) And we have the trusty ("deprecated but not really") remove-if-not to handle the verbosity of >>37189 Can't you type your lst parameter as list[String]?

>>37177 str.isupper is a function object str.isupper('ABCD') is a function call the point is, if you want to select out only uppercased strings, you can do list(filter(str.isupper, ['lorem', 'IPSUM', 'dolor', 'sit', 'amet'])) by just passing the function object to a function that calls it (filter will call the first parameter (function) on each element in the second parameter (list)) ability to pass a function to another function (or return a function from another function) is "higher order functions" effectively you give a function something to execute >>37189 >python does encourage a poor style maybe python just isn't a "oneliner" language, you get the choice of either writing - a procedural mess - beautiful OOP (rare) - beautiful FP but no nice way to write tiny oneliner functions like you could with ruby python, though, does lend itself to FP and function composition but you have to make an effort you can do much much more this way but you either have to implement your own tooling (see pipe, curry, complement below) or import an additional functional library (pypi toolz is often good enough) import functools import operator pipe = lambda *args: functools.reduce(lambda acc, el: el(acc), args) curry = lambda fn, *args: lambda *rest: fn(*args, *rest) complement = lambda fn, *args: not fn(*args) def rem_upper(coll:list[str]) -> list[str]: not_upper = curry(complement, operator.methodcaller('isupper')) return pipe(coll, curry(filter, not_upper), list) print(rem_upper(['lorem', 'IPSUM', 'dolor', 'sit', 'amet'])) # => ['lorem', 'dolor', 'sit', 'amet']

>>37208 > >python does encourage a poor style > maybe python just isn't a "oneliner" language, you get the choice of either writing ??? The comment about style is about typical python code which is a "procedural mess" like you outlined. Namely python is outright hostile to functional programming, both in the cultural use of it and from the language itself[1][2]. There are a few things that can lend themselves to writing code in a functional style 1. Tail calls 2. functional data structures 3. alternative methods of computation/algorithm management One does not to need fulfill all of these criteria to write in a functional style, but they do help encourage functional code. For example, Clojure lacks tail calls, but otherwise has 2, and relies upon HOF and other sorting means to write programs (->> "aaaabbbcca" (partition-by identity) (map (juxt (comp str first) count)))) This program was taken from [3] and sort of shows off some of the alternative tools to recursion or looping that is used. Clojure is known for it's functional style so this shows that one does not need to fulfill all of the criteria. Now, even languages like forth feel very functional. Although one bashes of bits quite often, the loop word in forth actually does not mutation but rather pushes an argument onto the return stack [4]. Another example is common lisp, TCO is covered by the spec but all common compilers modulo ABCL and ECL have tail calls so people often do assume it. CL's main data structure, the list, is also functional in that consing does not mutate the list, and there are commonly used libraries that provide other functional data structures [5] CL-USER> (defparameter *x* (list 1 2 3)) *X* CL-USER> (defparameter *y* (cons 5 *x*)) *Y* CL-USER> *y* (5 1 2 3) CL-USER> *x* (1 2 3) Great with that out of the way, I will now argue python has none of the qualities, and ease of functional abilities in the current ecosystem is worse than Java. Namely for point 1. you can refer to Guido's argument about tail recursion [2]. There are libraries that I know of that can transform python recursion into trampolines for you, however these are quite hcaky and I doubt serious projects would let you get away with using them. For 2. I could not find any functional data structures for python. At least on simple searches. Python has quite a large ecosystem of packages, yet it seems to me that there has been no development of functional data structures. So although you have posted some golfed code showing lambdas (we will get back to it), when working over collections that isn't just operating over all of it, you will be mutating the list >>> x = [1,2,3,4] >>> y = x.append(5) >>> y >>> x [1, 2, 3, 4, 5] thus functional sharing of any data structure within python does require deep copies, and functional style can not efficiently be maintained without rolling all your own datastructures for one of the biggest ecosystems in programming. For point 3, you have shown some functions with imports that give you HOF or otherwise iterating over things, and to be fair things like comprehensions exists, however even with all these tools python's general operations on this matter are quite lacking. Genernally speaking in languages that can allow for good functional style there are typically a plethora of functions one can go to, and by volume python lacks this. Another easy example are optional arguments which are trivial mutated, I've ran into this issue quite a bit when trying to write python functionally, it's a fairly big gotcha that one has to work around. Further for control flow mechanisms, python leans more heavy on the exception side for some basic functionality, which exhibit non local effects. I can go on for a while about this point, but I won't belabor it. Now lets get on with Java. I have written functional java before and it was much less painful than trying to write functional python. Java like python lacks 1, however it has 2, and a surprising amount of 3 in the control flow import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List; import java.util.LinkedList; import java.util.concurrent.ConcurrentSkipListSet; import java.util.stream.Collectors; import java.util.stream.Stream; import java.util.stream.IntStream; import java.util.Optional; public static Optional<Hand_Type> makeTwoPair( Optional<Integer> twoHigh , Optional<Integer> twoLow , List<Card> cards) { return findCard(cards, twoHigh) .flatMap(high -> findCard(cards, twoLow) .map(low -> { Stream<Card> cardWithPairsOut = cards.stream().filter(c -> c.getValue() != low.getValue() && c.getValue() != high.getValue()); List<Card> rest = grabHighest(1, cardWithPairsOut).toList(); return (Hand_Type) new TwoPair(low, high, rest.get(0)); })); } public static Optional<Hand_Type> makeThreeOfAKind(Optional<Integer> threeOfAKind, List<Card> cards) { return findCard(cards, threeOfAKind) .map(threeCard -> (Hand_Type) new ThreeOfAKind( threeCard , grabHighestArray(2, cards.stream().filter(c -> c.getValue() != threeCard.getValue())))); } public static Hand getBestHand(Player player, Card... cards) { Arrays.sort(cards); // Here int[] freqs = new int[13]; for (Card card : cards) { freqs[card.getValue() - 2]++; } // Four of a kind was here Optional<Integer> threes = threePair(freqs); Optional<Integer> twos = highTwoPair(freqs); Optional<Integer> secondTwos = lowTwoPair(freqs); List<Card> cardss = Arrays.asList(cards); //Return the values Hand_Type hand = makeStraightFlush(cardss) .orElseGet(() -> makeFourOfAKind(freqs, cardss) .orElseGet(() -> makeStraight(cards) .orElseGet(() -> makeFullHouse(threes, twos, cardss) .orElseGet(() -> makeFlush(cardss) .orElseGet(() -> makeStraight(cards) .orElseGet(() -> makeThreeOfAKind(threes, cardss) .orElseGet(() -> makeTwoPair(twos, secondTwos, cardss) .orElseGet(() -> makeHighCard(cardss))))))))); List<Card> bestHand = formRawBestHand(cardss, hand); return new Hand(player, hand, bestHand); } Although the code is quite verbose, one can write a fine functional style with this. I wrote this code a few years back rewriting our IRC's pokerbot in a better style. The code was never finished and you can see a loop still lives, but this shows off a few things. 1. Java has decent local control primitives if one uses them, namely optionals 2. the coverage of Java's are quite nice even having something like flatmap is quite useful This code does not show off using functional libraries nor data structures, this would have the same issue as python. However I have tried some functional java libraries and they work well, I have not used [5] but people seem to point to this for functional data structures and ideas, I forget what I've used. However, I hope this is somewhat sufficient to see that Java has greater functional capacity than Python. Note that we can make non functional languages look functional without too much effort, I can reimplment a lot of things from scratch, however we ought to recognize some languages are much more hostile towards this kind of work than others, and python is fairly up there on hostility. The saving grace of python is that since it's type system is much loser, you can get away with some ideas you can't in say go. Now I will cover your coding example. Why did you make pipe curry and complement lambdas, in normal style like you have for rem_rupper you should be using def. Python lambdas can not go past a single line and are fairly gimped [6]. Further functional programming is not about oneliners, often HOF are more compact and information is more compact, but that is not the point. Thus tools like pipe and curry are nice, but not necessary. They have their use, but the main point is rather functional transparency and data flow data structures is the main benefits, which the original python code with loops was quite horrible with.

>>37208 >>37210 Lastly I want to cover the Object Oriented style. Python's object system seems gimped. Python's object system is inspired from the MOP of CL but from what I've understand Guido decided to skimp out on many parts of the protocol, the practical upshot is you can do this >>> class MyClass: ... pass ... >>> def func(self): ... print("func") ... >>> MyClass.myMethod = func >>> x = MyClass() >>> x.myMethod() func However, you have the same linguistic issue, of having fractured syntax, one has to remember if something is written as a method or a function, as they do carry syntatic differences >>> [1].len() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'list' object has no attribute 'len' >>> len([1]) 1 better designed object systems typically avoid this issue and have a much better meta grounding when it comes to how Objects work (Smalltalk, Self, CmmonLisp). Thus I wouldn't really consider python as writing any kind of greatness when it comes to writing Object Oriented code (I don't even believe java can if you follow [7]) The main style of python is procedural with some minor OO influences, with an outright hostility to the functional paradigm. Due to these reasons, I typically avoid writing any python, and instead write code where my style is more welcomed. The set of languages where my style is more welcomed are in languages that I too would rather not write in but are more inviting, such as Rust or Java. [1] https://www.artima.com/weblogs/viewpost.jsp?thread=98196 [2] http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html [3] https://okmij.org/ftp/Algorithms/grasping-all-apples-at-once.html [4] https://www.forth.com/starting-forth/6-forth-do-loops/ [5] https://fset.common-lisp.dev/Site/index.html [6] https://www.artima.com/weblogs/viewpost.jsp?thread=147358 [7] https://www.cs.utexas.edu/~wcook/Drafts/2009/essay.pdfra

>>37208 >Python >aka the language without real closures and with a fat divide between statements and expressions (cf if, match) >beautiful FP

>>37210 >Why did you make pipe curry and complement lambdas i generally don't, toolz package provides these many more non-mutating functionalities >>37210 >functional data structures for python pyrsistent, have yet to try it myself >>37210 >functional programming is not about oneliners all i said is python lacks ways to write tiny oneliner functions that can match ruby's (generally used with OOP) terseness, as an example you have put a lot of effort (which i sincerely appreaciate by the way) in to proving that clojure and java are better at FP than python which i don't disagree with what i'm pointing out is that when i mostly work with python it is barely possible to write beautiful OOP so FP is the only alternative to procedural spaghetti messes python is hostile to FP, sure, but let's not pretend it's impossible to begin with >data structure within python does require deep copies, and functional style can not efficiently be maintained without rolling all your own datastructures for one of the biggest ecosystems in programming i avoid mutating state and have so far not had any problems with this >>> x = [1, 2, 3, 4] >>> y = x + [5] >>> y [1, 2, 3, 4, 5] >>> x [1, 2, 3, 4]

>>37214 > i avoid mutating state and have so far not had any problems with this >>> x = [1, 2, 3, 4] >>> y = x + [5] >>> y [1, 2, 3, 4, 5] >>> x [1, 2, 3, 4] If I were you, I'd avoid using + for lists >>> import time >>> def trace(f): ... t = time.process_time() ... f() ... return (time.process_time() - t) ... >>> trace(lambda: 3) 7.743000000004496e-06 >>> y = [y for y in range(100000)] >>> x = [y for y in range(100000)] >>> len(x) 100000 >>> trace(lambda: x + y) 0.0068357049999999975 >>> trace(lambda: x.append(1)) 8.935999999987176e-06 >>> trace(lambda: y + [1]) 0.0028086979999999984 >>> trace(lambda: x + [1]) 0.002196110000000001 >>> trace(lambda: x.append(1)) 1.4732000000017287e-05 Since python lists are really extendable arrays, and not lists, + is a O(n) operation, you can see if it you blow up the list size. Meaning if you try any basic functional algorithm, using + to extend the list, then your code would be very inefficient. At which point you ought to just create proper cons-cells. Data structures in python were not, and should not be used functionally due to just how bad this is. Functional data structures rely upon functional sharing. Let me give a small example >>> x = [1] >>> y = [2] >>> z = x + y >>> z [1, 2] >>> z.append(3) >>> z [1, 2, 3] >>> x [1] >>> y [2] >>> x.append(6) >>> x [1, 6] >>> y [2] >>> z [1, 2, 3] >>> y.append(5) >>> y [2, 5] Here we can see z is a completely new array that shares nothing in memory with x or y. However if we compare this to CL, which does let me mutate lists if I really wanted to [1]> (defparameter *x* (list 1 2 3)) *X* [2]> (defparameter *y* (cons 5 *x*)) *Y* [3]> (car *x*) 1 [4]> (setf (car *x*) 1000) 1000 [5]> *x* (1000 2 3) [6]> *y* (5 1000 2 3) As we can see here *x* and *y* share the list of *x* and a mutation on *x* is also a mutation on *y*! this means that consing is really O(1). Lists are just a basic data structure, but this applies to pretty much all data structures. it is a fairly big abuse to make ephemeral data structures to act like persistent ones because of the big O ends up being very bad. In other words, data structures in python were meant to be mutated. > FP is the only alternative to procedural spaghetti messes Sounds to me you want wishful thinking, a way of salvaging python. But why? Putting stake in a programming language that is hostile to styles you want to write hardly seems worth it. Use python as a tool if no other language makes it easy to do a task (this can happen due to libs, but I've never had to stoop to using python personally), and yeah abuse what you can to make it not soykaf. If you are at a company that is all python, then yeah try to help out. But realize in the latter you will have to deal with the zen of python and making code "pythonic" so your coworkers can understand the code. > python is hostile to FP, sure, but let's not pretend it's impossible to begin with I can write functional code in C if I wanted, no one is going to stop me. But yeah I can write functional code in almost any language, no claim of impossibility were had > >functional data structures for python > pyrsistent, have yet to try it myself With that said, it seems that pyrsistent would do the trick, though I'm unsure how well maintained it is, that might help

>>37229 i knew i'll regret mentioning list-to-list addition instead of toolz.concat https://toolz.readthedocs.io/en/latest/_modules/toolz/itertoolz.html#concat

>>37240 Even with your link it is not saved. Yeah you get O(1) concatenation, with making them both streams and chaining them, if yield is lazy [1] However this glosses over the fact of how one actually builds up recursive algorithms with lists, let me give an example CL-USER> (defun my-mapcar (f xs) (if (null xs) xs (cons (funcall f (car xs)) (my-mapcar f (cdr xs))))) MY-MAPCAR CL-USER> (my-mapcar #'1+ (list 1 2 3 4)) (2 3 4 5) Here I have a naive implementation of map, it is naive because it lacks the TCO transformation, nor does it accept multiple lists. With this simplified version we can see that we check if the list is null or cons, if it's cons, we then recurse on the tail (cdr). Now if we look at the library you posted, drop looks quite expensive [2] [3]: def drop(n, seq): """ The sequence following the first n elements >>> list(drop(2, [10, 20, 30, 40, 50])) [30, 40, 50] See Also: take tail """ return itertools.islice(seq, n, None) def islice(iterable, *args): # islice('ABCDEFG', 2) --> A B # islice('ABCDEFG', 2, 4) --> C D # islice('ABCDEFG', 2, None) --> C D E F G # islice('ABCDEFG', 0, None, 2) --> A C E G s = slice(*args) start, stop, step = s.start or 0, s.stop or sys.maxsize, s.step or 1 it = iter(range(start, stop, step)) try: nexti = next(it) except StopIteration: # Consume *iterable* up to the *start* position. for i, element in zip(range(start), iterable): pass return try: for i, element in enumerate(iterable): if i == nexti: yield element nexti = next(it) except StopIteration: # Consume to *stop*. for i, element in zip(range(i + 1, stop), iterable): pass which looks like we do O(n) work to take the tail, meaning if we were to transform the my-mapcar function into python, you should see serious slowdown from deconstructing the list. This is mostly due to streams in python being mutable, thus doing it properly functionally is at a O(n) cost, meaning that in all reality you'd want to use a loop or mutable work over it to avoid a O(n^2) my-mapcar [1] https://docs.python.org/3/library/itertools.html#itertools.chain [2] https://toolz.readthedocs.io/en/latest/_modules/toolz/itertoolz.html#drop [3] https://docs.python.org/3/library/itertools.html#islice

>>37254 Actually i may be incorrect, it could be O(1) due to yield semantics. But if you hand it back a stream it may just mutate it, if I'm reading the logic right, thus still O(n) conversion for safety if so

here's a book on fp in pyuthon it covers some of the stuff discussed in the thread

Python packaging is still a mess in early 2024: https://chriswarrick.com/blog/2024/01/15/python-packaging-one-year-later/ >>33751 Python might get JIT bytecode compiler in a future release: https://tonybaloney.github.io/posts/python-gets-a-jit.html

>>33751 Beginner question: What's the "pythonic" way to deal with the declaration of attributes that will not be assigned immediately at instantiation? I hope the question makes sense but using the 'classic' cars example: python class Car: def __init__(self, color, brand): color = self.color() brand = self.brand() This is fine if you will construct objects as newcar = Car(white, honda) But what happens if you want to add attributes sequentially based on user input? Such as, requesting the user to add the information himself in order to later manipulate the data in some way? I'm conflicted on whether the answer is: python class Car: def __init__(self, color, brand): color = None brand = None or fuarrrk it and just don't put anything in __init__ all together, but I feel that not explicitly declaring what attributes should be inside is wrong but at the same time, using some temporary variable to hold the data and later simply pass the arguments to the constructor that way doesn't feel right because I'm using more variables than I need. I tried looking this up online but either I'm a complete idiot or I should switch search engines again.

>>38352 class Car: color: Color # maybe an enum brand: Brand def __init__(self, data): self.data = data this is basically a glorified comment

>>37211 >>37210 >>37212 Fact that python as a language does not provide some features that FP-focused languages do is unrelated to python's ability to run programs written in an FP way. >paradigm in which function definitions are trees of expressions that map values to other values, rather than a sequence of imperative statements which update the running state of the program Can python compose pure functions? It can. Can python perform iterations, reductions, filtering, mapping, and any other higher-order functionality over scalar data or iterables? It can. Can python lazily evaluate an iterable through a sequence of functions or lambdas? It can. But, can Python do recursion? Barely. Not only does the TCO has to be taken care of manually (or via decorators) but there is also a recursion limit set in the language itself. A pain point, sure, but addressable. Can python compose a function which manually optimizes tail calls? It can. What is the problem in all of this? If you have to work with Python, for money, and if you understand or are interested in FP as a paradigm and can avoid writing gibberish that confuses your coworkers - you are more than able to leverage FP via some Python features in order to gain benefits. My workplace was in that position and we have gained tremendeous benefits from functional programming. One major unspoken point here that is overlooked is, as someone said (as opposed to implementing a recursive function like you would do in a Lisp), >that in all reality you'd want to use a loop or mutable work and what I want to stress that you are not incorrect or wrong somehow, all your points stand - but what you are arguing is totally beside the point. Of course tradeoffs are needed, so far I have encountered only one or two major ones. Everything else can be pragmatically done in a straightforward manner. Of course I'll have to use a loop or mutate state for TCO (ALL programming languages do this btw), in my case explicity, but so what?

>>33751 >GUIs > PyQt There's also PySide6, and frankly if you're working with Qt in Python, you might as well be using PyQtGraph, as it adds a lot of non-graph-related minor functions, and the graph widgets are easy enough to use that it's effectively required if you want to quickly graph anything. QWebEngine is also a useful tool as far as "make this webapp into a desktop app as fast as humanly possible" goes, especially relative to the other "chromium in a box" options. (I've used Electron before, it's...unfortunate.) I like using QThreads, they're a quick and easy method of inter-thread communication, the Socket/Signal system is more or less a highly-managed version of GoLang's channels.

I don't know where else to share this experience, but I'm going through "Python Concurrency with asyncio" by Matthew Fowler and while I do hope it gets better, last parts of 2nd chapter and so far all of the 3rd chapter I've gone through (some 70%) are a complete grind. Like I get the point and I get why it has to be the way it is, but it is a terrible reading experience. I have done socket programming before and understand the concept so it's only half difficult, but I expect someone who hadn't done any socket programming AND hadn't touched and concurrent programming will have a TERRIBLE experience going through this. Will update further as I go through the book.

>>39270 Same anon reporting in, at chapter 13 right now, got 13 and 14 left. It does get better, I anticipated correctly that all the fatiguing chapters in the beginning are fundamental building blocks for the chapters that come afterwards; and while subsequent chapters gradually get better (with some meandering around ch 8, 9 & 10 (streams, web apps & microservices)) the very core of the book seems to be the best - ch 4, 5, 6, 7. With ch 8 being something I'll have to revisit a bit later to figure out what I can make use of. While it's not really a breezy read I'm glad I put the effort to parse (if not grok) the book so far. nb as I'll post again soon when I'm done

This might be an odd question to ask, but I don't think I have a reasonable answer nor can I find one - how to stop mixing up dictionaries and objects in Python? Sometimes for example, and I noticed this frequently becomes an issue with using 3rd party libraries, I call a function A that returns an object but then I call function B on the same class/module that returns almost identical data as the object - except it's a dictionary. And then I end up with errors like AttributeError: 'dict' object has no attribute 'xyz' type hints help to a degree but with 3rd party code I cannot figure a way to be certain of exactly what a return value will be. Any ideas?

Hey all, am rewriting my imageboard to use starlette, uvicorn, and postgres and have run into an issue where was previously using a group_by to obtain a single image for each post for the catalog view. Would now like to return only the first file for each such post rather than what would be a random one in postgres but am having difficulty with my query in SQLAlchemy. It seems the generated SQL is correct, but SQLAlchemy is still throwing an error. Is there a better way to do this, and more importantly one that works? Thanks! python async def catalog_files (session, board): return await session.execute( select(File, func.row_number().over(partition_by=File.post, order_by=File.id).label("row_number")). where(File.board == board). where(text("row_number = 1"))) sql SELECT files.id, files.post, files.board, files."user", files.address, files.name, files.location, files.thumbnail, files.mime, files.size, files.spoiler, row_number() OVER (PARTITION BY files.post ORDER BY files.id) AS row_number FROM files WHERE files.board = 'b' AND row_number = 1 sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class'asyncpg.exceptions.UndefinedColumnError'>: column "row_number" does not exist

>>40454 well idk but this >where(text("row_number = 1"))) doing raw textual SQL instead of the proper python construct is a bad idea and can cause an invalid query idk about the generated sql either, but with SELECT files.id, files.post, [all that other soykaf], row_number() AS row_number FROM files , wouldn't the SELECTed row_number value be accessible as column row_number.row_number instead of row_number ? 1. don't use raw SQL when you have an ORM 2. try running the SELECT without WHERE, see what it outputs, why there isn't a row_number column in that output

>>40455 >Don't use raw SQL when you have an ORM Did try this of course, there is to my knowledge no other way to access the labeled column. Obviously this isn't very nice, but was operating under the assumption that it was a limitation of SQLAlchemy, or there was some hidden alternative (which querying here might reveal). The following syntax is used for comparison between literals: where("row_number" == 1) >try running the SELECT without WHERE, see what it outputs, why there isn't a row_number column in that output This simply works, the row_number output appears, the problem is in trying to access the column by label name. >>40455 >wouldn't the SELECTed row_number value be accessible as column row_number.row_number instead of row_number ? Was trying to use a label and not an aliased subquery/table. Attempting to use a subquery and alias made for an even uglier (but functional!) solution. python async def catalog_files (session, board): row_number = alias(select(func.row_number().over(partition_by=File.post, order_by=File.id).label("row_number"), File.id.label("id")).subquery(), name="row_number") return await session.execute( select(File). select_from(row_number, File). where(File.board == board). where(text("row_number.id = files.id")). where(text("CAST(row_number.row_number AS int) = 1"))) Cheers!

>>40456 The CAST didn't turn out to be necessary, my apologies for including it.

>>40456 >>40457 What was missing was the "c" notation for accessing columns. For completeness here's the finished query: python async def catalog_files (session, board): row_number = alias(select(func.row_number().over(partition_by=File.post, order_by=File.id).label("row_number"), File.id.label("id")).subquery(), name="row_number") return await session.execute( select(File). select_from(row_number, File). where(File.board == board). where(row_number.c.id == File.id). where(row_number.c.row_number == 1))

Newbie here, I know that tons of people reccommend python as a beginner programming language, but I dont get what makes it any better than any other programming language, or even why I should learn it in the first place besides it being "beginner-friendly". Should I still learn python anyways just for the sake of it?

>>40555 If you have no use for it then don't learn it. I need something to write quick throwaway scripts, and Bash is counter-intuitive and unintelligible for anything beyond a one-liner, so Python fills that niche for me.

>>40555 Python's claim to fame, and the reason it gets recommended to beginners as frequently as it does, is the speed at which you can get a program up and running. This is in part because of its large stdlib, and in part because the language itself is built in a way where you generally don't need to think about details that aren't relevant to what you're doing in the moment. Much of this programmer productivity comes at the cost of increased overhead, but for the use-cases Python was built for, that doesn't really matter. It's designed for situations where the problem is more important than the solution; for writing quick scripts and/or providing an interface for code written in other languages. >Should I still learn python anyways just for the sake of it? I say yes. It's a very common language, and you'll probably end up using it to at least some degree in The Real World. As an aside, if you're just starting out, there's a good chance that you'll bounce off of the first language you try, so you should shop around until you find something that clicks with you. I personally couldn't grok most CS concepts until I learned Scheme, and that was like my fourth language.

How to increase the performance of Python programs? Is there something similar to Effective C++ book series? Is PYPY the way to go? I would like to avoid using C or C++ extensions because I want to use only one language for my project. Also, where can I find some suitable example projects for a beginner? >>40555 >Newbie here, I know that tons of people reccommend python as a beginner programming language, The only reason is that Python is easier than some other programming languages, like C, C++ or Rust. But there are also other programming languages that are as good beginner languages as Python is. Examples of such langs are Ruby, Tcl, Perl, Lua and even Scheme. (Actually, Scheme, Tcl and Lua are even easier to learn than Python because there are simply less things to learn) >Should I still learn python anyways just for the sake of it? It's the most commonly used language for scripting after Bash. It makes sense to learn it because you may need it (especially if you are employed in this field). The other benefit is that you can easily find libraries for all sort of useful stuff because Python is so popular.

>>41613 I haven't been concerned with python optimization for about a decade, but back in say 2015 the first thing I would try is to cythonize the code. The main thing that was required there is to set datatypes so cython can use them. But if it really mattered we'd use an extension/binding.