Python 3 Thread

Python 3 general. Share resources, PDFs and books. Also discuss and share projects you have been working on. Beginner Questions are also welcome.

I made a script that uses the Lainchan JSON API to keep track of new messages in specified threads. It sends me a mail, whenever new messages are posted in the threads I specified. Feedback and Tips for improvement would be appreciated. The script can be founde here: https://git.fuwafuwa.moe/mananapr/python-scripts/src/master

>>14 It's readable, looks like a nice little piece of code to me. You might want to put the initialisation of the post count in a function, though. Another tip: since you just have to do stuff every 10 minutes, it might be smarter to set up a cron job to rund your script instead of your constantly running script with time.sleep(10). And for further reading it could be cool to look into aiohttp instead requests to get familiar with the async features of python3.5, not really useful for a small thing like this, but I think it's really fun to learn this stuff.

The server took too long to submit your post. Your post was probably still submitted. If it wasn't, we might be experiencing issues right now -- please try your post again later. Error information: What the fuarrrk is this? I'm tryna upload PDF's

>>33 >>33 same here The server took too long to submit your post. Your post was probably still submitted. If it wasn't, we might be experiencing issues right now -- please try your post again later. Error information: {"readyState":4,"responseText":"<html>\r\n<head><title>413 Request Entity Too Large</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>413 Request Entity Too Large</h1></center>\r\n<hr><center>nginx/1.10.3</center>\r\n</body>\r\n</html>\r\n","status":413,"statusText":"Request Entity Too Large"}

I tend to be more comfortable writing C++ code, but I can program in python. My biggest issue is the fact that you don't really know what type something is without looking at it for a while, whereas in C-languages you know for a fact that int a will always be an int... How do you get around this? Do you just get used to it? Also, what programming environment do you lains use? I use emacs+bash. I can't seem to post pdfs atm, but once the servers are up and running, I'll put my fair share of knowledge

uploads werk again that's cool

How stupid is this mini sqlite wrapper? Any comments appreciated. Python #!/usr/bin/env python3 import sqlite3 import os from collections import OrderedDict class Database: """ bundle function for less soykafty database-interaction """ def __init__(self, db_path, setup_query=None): self.db_path = db_path if setup_query: if os.path.isfile(db_path): os.remove(db_path) self.apply_query(setup_query) def apply_query(self, query, payload=(), close_conn=True): conn = sqlite3.connect(self.db_path) c = conn.cursor() c.execute(query, payload) #eval conn.commit() #apply if close_conn: c.close() else: return c def read(self, query, fetch_number=1, result_abstraction=None, payload=()): # retun dict must be ordered dict c = self.apply_query(query, payload=payload, close_conn=False) assert c, "Error in read" data = c.fetchmany(fetch_number) # TODO: possible performance implications of fetchmany? c.close() # if we don‘t just want to return a list of tuples, but a list of OrderedDicts representing the data # (we use OrderedDicts to be able to map the postitoin of a tuple element to a key of a Dict in a convenient manner) if type(result_abstraction) == OrderedDict and len(data) > 0: assert len(data[0]) == len(result_abstraction), "wrong lenght of given result abstraction" result = list() for data_tuple in data: current_result = result_abstraction.copy() for i, key in enumerate(current_result): current_result[key] = data_tuple[i] result.append(current_result) return result return data """ example.png """ if __name__ == "__main__": db = Database("yomoda.db", setup_query="CREATE table quotes (quote text, dankness integer)") db.apply_query("INSERT INTO quotes VALUES (?, ?)", payload=("Good will come to those who wait", 9001)) # le memez implying that 2010 xD xD (kill me plz ;___;) results = db.read("SELECT * FROM quotes WHERE dankness > 9000", result_abstraction=OrderedDict([ ("quote", ""), ("dankness", 0) ])) for result in results: print("The dankness-level of the quote '{}' is: {}".format(result["quote"], result["dankness"]))

flood control I am angry

>>24 I thought of using cron initially, but switched to time.sleep() so that the script would also work on systems that don't have cron like Windows. And thanks for recommending aiohttp. It looks cool >>36 >My biggest issue is the fact that you don't really know what type something is without looking at it for a while, whereas in C-languages you know for a fact that int a will always be an int... How do you get around this? Do you just get used to it? Strangely, I never have felt this way. Maybe you'll get used to it. >Also, what programming environment do you lains use? I use vim+bash in a tiling wm.

>>48 >>49 >>50 >>52 >>54 >>55 I'll dump a few more once the uploads get fixed again.

I started learning python3 with 'Automate the boring Stuff with Python' and got to learn some cool modules like BeautifulSoup and Selenium. Now I want to get started with Machine Learning in Python. Can anyone point me to some tutorials or books? I don't know where to start

I've been working recently using a lot of map, filter and reduce functions inside Python and everything gets messy so easily. Do you know of any built-in solution for this? Several languages have solved this problem of composing functions by different aproaches. For example Rust has all those functions as methods for the Iterator trait then you can concatenate easily several maps/folds/filters. Elixir has the pipe operator. Do you believe that implementing a solution for this is worth it?

>>67 It's alright if you delete them. Do you know what's wrong with uploading? It works on and off and gives the error in >>35

>>71 Not yet, keep an eye on /q/ for updates concerning this.

>>70 Even though Python has map, filter, etc., I think the "Pythonic way" (I don't really like that phrase, personally) is to use list comprehension syntax, and the itertools and functools libraries.

>>152 More likely, to use for loops. Guido really doe snot like functional programming, hence why it's such a pain in Python.

I've been working on Driftwood 2D, a tiling game engine. After a long break I got back into it, and released a new version a few days ago. It's in Alpha and I've just started writing the User's Manual, so you can't expect too much from it, but if anyone wants to play around reading the test data will tell you how to get started. The engine is its own API; scripts from the game can call any public function in the engine. So, the source code is the API reference currently, each important function being already well documented. Features: * Infinite graphical and walkable layers * Resource caching * Virtual filesystem for game data and patches, supporting zip files * Tile and sprite animations * Sound effects and music * Configurable input handling * Timed callbacks * Lightmaps * Rudimentary database for saving data between plays * Simple widgets with TTF font support * Fully scriptable in Python 3 Requirements: * SDL2 * SDL2_image * SDL2_mixer * SDL2_ttf * Python >= 3.3.3 * Python jsonschema (https://pypi.python.org/pypi/jsonschema) * Python PySDL2 (https://pypi.python.org/pypi/PySDL2/) GitHub: https://github.com/seisatsu/Driftwood Latest Release: https://github.com/seisatsu/Driftwood/releases/tag/Alpha-0.0.4a

>>14 >Feedback and Tips for improvement would be appreciated. - The json fetch and load is textually repeated instead of being factored out into a function. - Count is initialized to 0 then incremented for every item in a sequence, so check out the len [1] built-in. - You aren't using the "__main__" [2] test to protect execution. - You are printing "Started Watching" every time. - There's no way to exit cleanly. - You are looping over the items rather than the keys in the initialization loop, even though the old count is unused. - You don't need end-of-line semicolons in Python. - In the send_mail call urls[url] - diff is the value you have in c. - In the send_mail body use string formatting [3] instead of concatenation in the url line to get the str calls for free. - Use a ternary if to get "post(s)" exactly right. [1] https://docs.python.org/3/library/functions.html#len [2] https://docs.python.org/3/tutorial/modules.html#executing-modules-as-scripts [3] https://docs.python.org/3/tutorial/inputoutput.html#fancier-output-formatting

>>166 Have you thought of making it into a fantasy console type thing like pico8 only Open-Source and running python?

>>225 I've thought of adding an in-game console where you can run Python code on the fly to interact with the API, but I didn't think of allowing an entire game to be developed in-game. It's an interesting idea, but the problem is that the map editor is a separate program so I couldn't integrate it unless I made an in-game map editor. It's a thought for the future.

>>257 cont... Another thought though is that once we handle mouse input you could just implement an entire map editor in game scripts and run it on the base engine. That's how powerful the scripting is.

>>140 I have the physical copy of that book and currently I'm on chapter 8. Everything up untill chapter 6 was pretty well done, but after that it seems that it's going downhill. There are too many code errors and the explanations seems rather too vague.

>>70 Python is utterly uncomposable. Weird lambdas and stupid distinctions between what's a statement and what's an expression, Because Guido Said So. Unfortunately there's no way around it that isn't a huge pain to implement, it's probably better to use Coconut or Hy.

>>13 wasted more time than i'm willing to admit trying to debug a program because i didn't know for loops in python are a lie

I made this abomination because Reddit's Atom feeds are too cucced for xscreensaver to parse It functions both as a CGI script (?r=<subredditname>) or as a standalone executable (subreddit name as first arg) https://github.com/JamesTheAwesomeDude/RedditAtom/blob/master/RedditAtom.py (Lainchan chokes on muh tabs)

>>451 What do you mean with this exactly?

>>525 I think the lainon refers to the fact that loops have no special scope in python (unlike e.g. C or Java) Example: python i = 12345 for i in range(43): pass >>> print(i) 42 as opposed to: c int i = 12345; for (int i = 0; i < 43; ++i) printf("hi dont optimize away kthxbye\n"); printf("%d", i); /* prints 12345 */

>>526 also list comprehensions behave differently in python3 and don't "leak" the variable (http://python-history.blogspot.de/2010/06/from-list-comprehensions-to-generator.html)

>>526 >I think the lainon refers to the fact that loops have no special scope in python (unlike e.g. C or Java) That's wrong and you should stop giving advice if you don't know how python scoping works. This issue comes up every single time someone mentions it and every wannabe(you) gets it wrong 100% of the time.

>>532 Why wouldn't you explain why it's wrong then

>>532 https://docs.python.org/3.6/reference/executionmodel.html >A scope defines the visibility of a name within a block. If a local variable is defined in a block, its scope includes that block. >A block is a piece of Python program text that is executed as a unit. The following are blocks: a module, a function body, and a class definition. [...] (*not* loops; except for list comprehensions since they "[...] are implemented using a function scope.") >The following constructs bind names: [...], for loop header >If a name is bound in a block, it is a local variable of that block Serious question, I don't really get why you say that this is wrong since the provided examples also work as expected. Additionally, like every StackOverflow answer on this would be incorrect then (e.g. http://stackoverflow.com/questions/6167923/block-scope-in-python) etc., so it would be kind to elaborate since, assuming you're right, it'd be a pretty common misconception. Thanks.

This script I wrote generates a table of all possible IPv4 subnets. It was a great exercise to improve my understanding of network addressing. #!/usr/bin/env python import math def subnet_info(network_bits): max_hosts = pow(2, 32 - network_bits) - 2 dec_octets = [] full_octets = math.floor(network_bits / 8) remaining_octets = 4 - full_octets for _ in range(full_octets): dec_octets.append(255) partial_octet = 0 for binary_digit in range(network_bits % 8): partial_octet |= (128 >> binary_digit) dec_octets.append(partial_octet) remaining_octets -= 1 for _ in range(remaining_octets): dec_octets.append(0) subnet_mask = ".".join([str(octet) for octet in dec_octets]) binary = " ".join([format(octet, "08b") for octet in dec_octets]) return (subnet_mask, max_hosts, binary) if __name__ == "__main__": rows = [["SUBMASK", "BITS", "HOSTS", "BIN"]] row_widths = [len(column) for column in rows[0]] for network_bits in range(8, 31): subnet_mask, max_hosts, binary = subnet_info(network_bits) network_bits = str(network_bits) max_hosts = str(max_hosts) for i, value in enumerate([subnet_mask, network_bits, max_hosts, binary]): if len(value) > row_widths[i]: row_widths[i] = len(value) rows.append([subnet_mask, network_bits, max_hosts, binary]) for row in rows: for i, column in enumerate(row): width = row_widths[i] if i in [0, 3]: print(column.ljust(width), end=" ") else: print(column.rjust(width), end=" ") print()

>>568 >full_octets = math.floor(network_bits / 8) Check out the // operator. >for [...] dec_octets.append Check out list * int and list.extend. >partial_octet |= (128 >> binary_digit) You don't need a loop. You can obtain x bits in the high position by shifting them up from the low position. You can obtain x bits in the low position by shifting 1 by x then subtracting 1. >if len(value) > row_widths[i]: Check out the max built-in. >if i in [0, 3]: In general, use sets rather than lists for membership testing, for that logarithmic feeling. >ljust >rjust The format language will do this for you. >>> data = [("Name", "Cost"), ("apples", 40), ("grapes", 8), ("pineapples", 100)] >>> c1 = max (len (x [0]) for x in data) >>> c2 = max (len (str (x [1])) for x in data) >>> fmt = "{:<{c1}} {:>{c2}}" >>> print ('\n'.join (fmt.format (*x, c1=c1, c2=c2) for x in data)) Name Cost apples 40 grapes 8 pineapples 100 >>> fmt = "{:>{c1}} {:<{c2}}" >>> print ('\n'.join (fmt.format (*x, c1=c1, c2=c2) for x in data)) Name Cost apples 40 grapes 8 pineapples 100 >>> Move your main block into a main method and call main() in the "__main__" if. That way you can be imported into the interpreter without execution, and executed afterwards if desired by invoking main().

I came across this snippet in python and am absolutely mystified as to how it works. Supposing that types is an array and x is a list of strings equal in length to types. Each entry in x corresponds to an entry in types and only one will be a non-empty string. the next() call returns the index of the non-empty element in x, which is then used as an index for types. types[next(i for i, e in enumerate(x) if e)] As far as I can see next() takes an iterable object and optionally a default value to return if the end of iteration is reached. The arguments given here really don't make sense to me, and yet it works.

>>14 Rather than keeping this process running forever and checking every 10 minutes, set up a cron job to run every ten minutes and have your code run once.

annoyance: if you have a bytes object, you can't iterate through it with a for loop, because it gives you an int rather than a byte.

>>601 >The arguments given here really don't make sense to me, and yet it works. I guess the confusing part is that it looks like there are two arguments but there is really just one argument, a generator expression. Maybe it becomes more clear without tuple unpacking: types[next(t[0] for t in enumerate(x) if t[1])]

>>601 >The arguments given here really don't make sense to me With square brackets you would have a list comprehension: [i for i, e in enumerate(x) if e] that would compute all values and return them in a list. You could then find the first one at index 0. With parentheses you have a generator comprehension: (i for i, e in enumerate(x) if e) which returns an iterator that will compute its values one by one when they are requested. With one next call you get the first value and don't compute the rest. The final piece is that when a generator comprehension is the only argument in a function call you would have two sets of nested parens, and python allows you to omit one set. Why? Because the BaDFaiL said so. Here's an example. Let's say patterns is a list of compiled regular expression patterns and input is a string to parse. Then you would do: result = next ((m for m in map (lambda p: p.fullmatch (input), patterns) if m), None) to stop checking at the first successful match. >>616 Would you prefer iterating over a list to yield lists of length 1 rather than the elements themselves? Didn't think so.

>>617 >>620 Thank you so much. I had spun off in the wrong direction and had been bothered by it all day. I had no idea about the lone generator being able to omit nested parens. I'm not sure saving two characters is worth making things less clear, but it probably makes no difference to people who are more experienced.

>>571 Thank you for taking the time to give me feedback, I've revised my program based on it: https://w1r3.net/RXrjQd.txt >You don't need a loop. You can obtain x bits in the high position by shifting them up from the low position. You can obtain x bits in the low position by shifting 1 by x then subtracting 1. I implemented this like so: partial_octet = 0xff & (0xff << (8 - network_bits % 8)) The AND with 0xff is necessary to ensure the number doesn't grow beyond eight bits, since shifting in Python won't make bits „fall off“. I've also decoupled the data generation and presentation by splitting the logic in the main function into a print_table and a subnet_rows function. The way you suggested to find the longest columns using max() rubs me the wrong way as it needs more iterations than the „less elegant“ solution. This isn't a big deal for such a small amount of data, though I would prefer the most efficient solution possible.

>>571 >>644 Addendum: Ignore the superfluous for loop on line 13, I've replaced that with dec_octets = full_octets * [255] .

Python >>> from ch1progs import * >>> ToDate(11, 19, 1995) 0 >>> def td(m, d, y): return (m << 12) | (d << 7) | (y % 100) ... >>> td(11,19,1995) 47583 ##### ToDate's definition in the source file def ToDate(m, d, y): return (m << 12) | (d << 7) | (y % 100) can someone explain this to me?

>>661 Pastebin the entire ch1progs file.

>>661 ToDate is a function that compacts a date into a number... maybe. n << m shifts n bitwise "to the left" m times. So, a binary number like 0101 bitshifted left 3 times would be like 0101000. n % m is modulo or the remainder for n divided by m. n | m combines two numbers with bitwise OR. So, if either bit in the same spot of both numbers is set to 1 the corresponding bit in the result is 1, otherwise 0. For example, 0101 | 0011 is like this: 0101 0011 ------ 0111 The idea of this is to combine multiple numbers together using a number. A simpler example of this would be to store two numbers under 8 together like this: def combine(a, b): return (a << 4) | b a is padded to give enough space for b, and then combined with b with OR. Anyway it's weird because I don't know why the year is being taken mod 100. Maybe we know the year is in the 1990s? Shifting the date to the left 7 spots gives 2^7 possible years, which is only 128, but given that the year is being taken mod 100 it makes sense. The other paddings make sense, as 2 ^ (12 - 7) is 32 and there are at most 31 days in a month.

>>672 I should also mention that this is a low level way of representing dates and not for Python. Python has one of the best date/time libraries around in the standard library (datetime), so in a real application use that. But if such a library didn't exist, you'd want to make it more like this: def to_date(m, d, y): return (m, d, y) ...or this: class Date(object): def __init__(self, m, d, y): self.m = m self.d = d self.y = y ...or this (which is what I'd do if I really had to) from collections import namedtuple Date = namedtuple('Date', 'm, d, y') The bitwise strategy of storing numbers is more commonly found in languages like C, but even then only when memory constraints are really tight (i.e., not in programs that'd be run even 10 or 20 years ago on PCs, much less today). Also as a nitpick, functions in Python by convention start with an uncapitalized letter while classes are CamelCase.

>>672 >>676 Lain was asking about the 0 result, obviously.

>>661 ToDate isn't doing what you say is in the source, since its returning 0 for y=1995, which can't happen with the definition you give (because of the OR). Either that or ToDate is changed later in ch1progs, so please do >>671

>>644 >>645 >https://w1r3.net/RXrjQd.txt >partial_octet = 0xff & (0xff << (8 - network_bits % 8)) This way is fine as well, the point was to get rid of the loop. My suggestion was: ((1 << x) - 1) << (8 - x) which doesn't need a final masking because it generates exactly the needed bits. >it needs more iterations Traversing the data matrix in either row major or column major order produces the same number of data accesses. You also get the same number of length comparisons. >lengths = [len(row[index]) for row in rows] >column_widths[name] = max(lengths) You don't need to store the lengths and compute the max afterwards. You can use a single pass [1]: column_widths [name] = max (len (row [index]) for row in rows) You can also do the entire length computation in one go: lengths = [max (len (row [col]) for row in rows) for col in range (column_count)] ># O(2nc)? Order of growth notation and analysis both ignore multiplicative constants. >column_widths = {} Dictionary lookup by key is much slower than list sccess by index. You only need the dictionary for the format call, so you can compute the lengths in a plain list, by column index, and create the dictionary afterwards. This should be a dictionary comprehension [2] combining the length and label lists using the zip [3] built-in. [1] https://docs.python.org/3/reference/expressions.html#generator-expressions [2] https://docs.python.org/3/reference/expressions.html#dictionary-displays [3] https://docs.python.org/3/library/functions.html#zip

>>690 https://w1r3.net/kAGqcY.txt I feel like I'm seriously overdoing it with the list comprehensions. >((1 << x) - 1) << (8 - x) That's clever. It didn't occur to me to mix in decimal arithmetic. >You don't need to store the lengths and compute the max afterwards. You can use a single pass [1]: Whenever I contract long expressions like that, I get the feeling that the readability of my code suffers. Am I just imagining that? I find it much easier to read code when variable names indicate what role a value plays in a complex expression.

>>761 >Whenever I contract long expressions like that, I get the feeling that the readability of my code suffers. >I find it much easier to read code when variable names indicate what role a value plays in a complex expression. >I feel like I'm seriously overdoing it with the list comprehensions. They can be switched to generator comprehensions when they are single use throwaway sources of data, which is the case in most of your program. As for comprehensions in general, you'll have to find your own balance, whenever efficiency arguments don't impose one choice over another. >https://w1r3.net/kAGqcY.txt One thing left to fix. The zip dictionary is recomputed by exactly the same computation for every row in join's comprehension, which is wasted work. This can also be seen at a purely syntactic level by noticing that it doesn't use the row value from join's comprehension. Therefore, the dict should be computed once, before the print call.

>>766 Done. Thank you very much for your feedback!

>>70 Instead of: map (sunglasses, map (commando, map (cravefish, penguins))) you can do: def rcompose (*funs): def ret (*args): fit = iter (funs) base = next (fit) (*args) return reduce (lambda acc, f: f (acc), fit, base) return ret map (rcompose (cravefish, commando, sunglasses), penguins)

>>438 >>140 So, like I said in my previous post, I have the physical copy of the book and I've found too many errors. You can however report errors on the oreilly website (which I did) http://www.oreilly.com/catalog/errata.csp?isbn=0636920028659 http://www.oreilly.com/catalog/errataunconfirmed.csp?isbn=0636920028659 BUT it seems like they've cleared up their "unconfirmed error" section and my reported errors were totally wiped (just like many other reports) and they're not even on the "confirmed" section. Wouldn't recommend buying this book at all/10

One of the current Python 3 projects I have been working on is https://github.com/Appleman1234/safeandsound , which is a backup script. python import configparser import errno import glob import gnupg import os import shutil import sys import tarfile import time from distutils.dir_util import copy_tree from subprocess import Popen def make_sure_path_exists(path): try: os.makedirs(path) except OSError as exception: if exception.errno != errno.EEXIST: raise def make_tarfile(output_filename, source_dir): with tarfile.open(output_filename, "w:gz") as tar: tar.add(source_dir, arcname=os.path.basename(source_dir)) def main(): config_filename = 'backup.cfg' if len(sys.argv) > 1: config_filename = sys.argv[1] # Read in variables from configuration file config = configparser.ConfigParser() config.read(config_filename) backup_root_directory = config.get('settings', 'backup_root_directory') scp_users = config.get('settings', 'scp_users').split(',') directories_to_backup = config.get('settings', 'directories_to_backup').split(',') tar_gpg_passphrase = config.get('settings', 'tar_gpg_passphrase').split(',') dump_gpg_passphrase = config.get('settings', 'dump_gpg_passphrase').split(',') encrypt_dump = config.getboolean('settings', 'encrypt_dump') encrypt_tar = config.getboolean('settings', 'encrypt_tar') scp_directories = ["/home/" + x for x in scp_users] scp_directory_dict = dict(zip(scp_users, scp_directories)) backup_sub_directory = str(int(time.time())) backup_directory = backup_root_directory + backup_sub_directory backup_directory_slash = backup_root_directory + backup_sub_directory + "/" tar_filename = backup_sub_directory.rstrip("/") + ".tar.gz" tar_filepath = backup_root_directory + tar_filename gpg = gnupg.GPG() # Create backup directory make_sure_path_exists(backup_directory) # Copy files into directory for directory in directories_to_backup: directory_suffix = os.path.basename(os.path.normpath(directory)) copy_tree(directory,backup_directory_slash + directory_suffix) # Database dump #os.system("mysqldump --all-databases > " + backup_directory_slash + "dump-$(date +%Y-%m-%d_%H-%M-%S).sql") mysqldump_process = Popen('mysqldump --all-databases > ' + backup_directory_slash + 'dump-$(date +%Y-%m-%d_%H-%M-%S).sql', shell=True) mysqldump_process.wait() dump_filename = glob.glob(backup_directory_slash + 'dump-*.sql') dump_gpg_filename = "" # Encrypt dump with gnupg if encrypt_dump: dump_gpg_filename = dump_filename[0] + ".gpg" dump_file = open(dump_filename[0], "rb") gpg.encrypt_file(dump_file, recipients=None, symmetric='AES256', passphrase=dump_gpg_passphrase, armor=False, output=dump_gpg_filename) os.remove(dump_filename[0]) # History #os.system("history > " + backup_directory + "history") shutil.copy("/root/.bash_history", backup_directory_slash+"history") # Make tarball #os.system("tar -czv " + backup_directory + "-f " + tar_filepath) make_tarfile(tar_filepath,backup_directory) # Remove backup directory #os.system("rm -rf " + backup_directory) shutil.rmtree(backup_directory) filename = tar_filename # Encrypt tar with gnupg if encrypt_tar: tar_gpg_filename = tar_filepath + ".gpg" tar_file = open(tar_filepath, "rb") gpg.encrypt_file(tar_file, recipients=None, symmetric='AES256', passphrase=tar_gpg_passphrase, armor=False,output=tar_gpg_filename) os.remove(tar_filepath) filename = tar_gpg_filename #os.system("cp " + gpg_file + " " + scp_directory) for user in scp_directory_dict: directory = scp_directory_dict[user] # Copy to scp directory shutil.copy(filename,directory) # Adjust permissions os.system("chown " + user + ":" + user + " " + directory + "/" + tar_filename) if __name__ == '__main__': main() another is plugins for the IRC bot located at https://github.com/Appleman1234/arisu , which based on Err (http://errbot.io/en/latest/). Pull requests and feedback are welcome.

Currently using Python for an extra credit numerical simulation project for my intro to Electricity and Magnetism class. I'm kind of amazed at how powerful python. Here's the code that runs, and just as a test, the attachment is a .py file with some of the methods I use below: import matplotlib.pyplot as plt import math import numpy as np import Project3 as P3 #Conductors look like regions where phi is uniform and unchanging, #which we can do by including it in the boundary! N = 100 boundary_vals = np.zeros((N,N)) #Boundary condition values b = np.ones((N,N)) #Indicates points on the boundary charge = np.zeros((N,N)) b[2:N-1,2:N-1]=0 #now only the points on the edges have value 1 #line """ bv[48:52,40:60]=ones(5,21) b[48:52,40:60]=ones(5,21) """ #square: """ bv[40:60,40:60]=ones(21,21) b[40:60,40:60]=ones(21,21) """ #circle: """ aradius=20 for i=-radius:radius, j=-radius:radius if (i^2+j^2)<radius^2 bv[i+50,j+50]=1 b[i+50,j+50]=1 end end """ #plus: l1=16 l2=2 b[50-l1:50+l1,50-l2:50+l2]=1 b[50-l2:50+l2,50-l1:50+l1]=1 boundary_vals[50-l1:50+l1,50-l2:50+l2]=1 boundary_vals[50-l2:50+l2,50-l1:50+l1]=1 iterations = 1000 #needle """ height=40 for i in xrange(height): for j in xrange(height/2): if(abs(j-height/4)*10<i): b[50-height/2+i][50-height/4+j]=1 boundary_vals[50-height/2+i][50-height/4+j]=1 """ (phi, delta) = P3.Relax(boundary_vals, b, charge, .1, iterations) plt.figure(1) plt.title("Electric Potential") plt.pcolormesh(phi) eField = P3.ElectricField(phi, b,.1) plt.figure(2) plt.title("Electric Field") print "Is this working?" E_Field_Magnitude = np.zeros(shape=(len(eField), len(eField[0]))) for row in xrange(len(eField)): for col in xrange(len(eField[0])): E_Field_Magnitude[row][col] = math.sqrt(eField[row][col][0]**2 + eField[row][col][1]**2) plt.quiver(eField[:,:,0], eField[:,:,1], E_Field_Magnitude) plt.show()

>>915 >xrange >print statement >>Python 3 Thread

>>917 Wait, is xrange really not supported in Python 3? Besides those couple of things, it's pretty much Python 3 though. I'm kind of curious, why all the drastic changes between Python 2 and 3?

>>919 >is xrange really not supported in Python 3? https://docs.python.org/3/whatsnew/3.0.html#views-and-iterators-instead-of-lists """range() now behaves like xrange() used to behave, except it works with values of arbitrary size. The latter no longer exists.""" >why all the drastic changes between Python 2 and 3? Because Guy the Opossum said so. https://docs.python.org/3/whatsnew/3.0.html """There are more changes than in a typical release, and more that are important for all Python users. Nevertheless, after digesting the changes, you’ll find that Python really hasn’t changed all that much – by and large, we’re mostly fixing well-known annoyances and warts, and removing a lot of old cruft."""

>>912 The body of main() is a bit overloaded. Smaller functions with clearly defined tasks would be easier to read and edit. >backup_directory = backup_root_directory + backup_sub_directory >backup_directory_slash = backup_root_directory + backup_sub_directory + "/" It's preferable to use the os.path.join() function for path concatenation, even if your code is not supposed to be OS-agnostic. It communicates more clearly to readers of your code that you're dealing with file system paths. >dump_gpg_filename = "" >if encrypt_dump: > dump_gpg_filename = dump_filename[0] + ".gpg" If statements don't have scopes in Python. (AFAIK only classes, functions and modules do.) Pre-defining variables like this isn't necessary. https://docs.python.org/3.4/reference/executionmodel.html#resolution-of-names >os.system("chown " + user + ":" + user + " " + directory + "/" + tar_filename) This can be done without a shell invocation using shutil.chown. https://docs.python.org/3.3/library/shutil.html#shutil.chown

What editor do most Python programmers use? Do people use dedicated Python editors like SPE, Eric, Boa Constructor? Since Python is indentation delimited are there special features that allow you to insert code by reindenting large sections of code? Are there utilities to prettify (auto-indent) Python code?

>>1067 I was in the middle of installing Python plugins for Atom when I realized that I was attempting to make a worse version of Pycharm. That's what I use now.

>>1067 I use Vim, and I make heavy use of visual select and the < > commands to indent lines and blocks when editing Python code.

Is there any canonical book for learning Python 3 with prior programming experience (1 year of CS in Java)? A lot of the recommended books are for complete neophytes but I really don't need that much explanation of what an "int" is and so on. I've been reading the book >>140 linked which seems fine but I'm not sure if there's anything better.

>>1143 the introduction on the site is actually really good and covers everything you'll see in code besides decorators. But if you've used java they're the same thing, except how you write them.

>>1143 The following recommendations are based on a conversation in the IRC, which was asked by two people that were beginners to Python. My first recommendation is the first attached PDF, thought I didn't learn from this book myself personally. The supporting documenttion for the first recommendation is located at https://www.qtrac.eu/py3book.html . My next two recommendations are https://leanpub.com/python_101/c/48hours and https://leanpub.com/python201 though I didn't use these resources personally. Resources I learned from personally include https://www.diveintopython3.net/ (third attached PDF) https://docs.python.org/3/tutorial/index.html and the second attached PDF. Other lists of books include https://wiki.python.org/moin/IntroductoryBooks and http://python-guide-pt-br.readthedocs.io/en/latest/intro/learning/ and http://python-guide-pt-br.readthedocs.io/en/latest/. As for your specific question I recommend the second attached pdf, e.g. http://greenteapress.com/thinkpython/thinkpython.pdf because you can compare and contrast it with its Java version at http://www.greenteapress.com/thinkapjava/thinkapjava.pdf . Also compare and contrast the list of resources located at https://wiki.python.org/moin/BeginnersGuide/Programmers with https://wiki.python.org/moin/BeginnersGuide/NonProgrammers Other recommendations theads on the internet also recommend the following books, though I haven't read them all yet personally Effective Python, Python Cookbook and Fluent Python and Writing Idiomatic Python. Finally another resource I mentioned in that IRC conversation were the GUI tutorials at zetcode.com e.g. https://zetcode.com/gui/pyqt5/ https://zetcode.com/wxpython/ https://zetcode.com/gui/pygtk/ https://zetcode.com/gui/pysidetutorial/

>>1159 Would you (or anyone else) have a recommendation for someone who has written a decent amount of python, but is interested in becoming more familiar with the language and improving proficiency?

>>1160 https://lainchan.org/%CE%BB/res/83.html#937

>>1160 From the third last line of my previous post, please find the attached files, please note the Effective Python was already posted in another thread and linked by >>1174 . First attachment is Python Cookbook , also available at https://learko.github.io/books/coding/Python_Cookbook_3rd_Edition.pdf and the source code and supporting stuff is available at https://github.com/dabeaz/python-cookbook Second attachment is Fluent Python also available at https://1.droppdf.com/files/X06AR/fluent-python-2015-.pdf and source code and supporting stuff available at https://github.com/fluentpython Third attachment is Writing Idiomatic Python. Errata is https://github.com/jeffknupp/writing-idiomatic-python Also see https://wiki.python.org/moin/AdvancedBooks and https://github.com/vinta/awesome-python and https://python.libhunt.com/

I'm not sure if I understand memoryview 100%. A memoryview object gives access to the data stored in a bytes, bytearray, or some other object that implements the buffer protocol like array.array, without actually making a copy of the data. memoryview objects are always the same size (192 bytes on my machine), so the idea is to use memoryview when you want to take a big slice from another object. Using memoryview instead of taking a slice directly will save RAM. Am I missing anything? Are there other applications of or reasons to use memoryview?

>>1371 >when you want to take a big slice >other applications of or reasons to use memoryview As a general principle, python uses (one-dimensional) memoryviews to avoid the necessity for functions that deal with bytes-like objects to have overloaded versions that also take a starting offset and a length. In effect, a memoryview acts like a (backing store, starting offset, length) triple -- what java calls a buffer. Beyond the consumer scenario that you describe, there is also the producer scenario where a method will give you some data and takes a storage object as an argument for this purpose, but not an offset and length, such as the readinto type methods. When you have some partial content in store and want new content appended to it, you pass the producer method the appropriate memoryview that covers your still empty storage area. There are also some esoteric uses like reinterpreting a one-dimensional array as a multi-dimensional array and vice versa, but these will only show up very rarely. Also, remember that memoryviews are context managers and, unless you have a good reason not to, their use should be wrapped in a with block.

>>1374 Cool. I didn't know memoryviews can be wrapped in a with block, thank you anon.

>>1279 Thanks a ton for this!

I am currently making an automated CMS - like program (think like a Wordpress but flat - filed and local) for my local file. I hate the idea of manually placing my files (for music, images, memes, any file). Just today I decided to make a Python script to be run via Cron to automatically move and rename files in a folder. So I have just need to put every file in a folder with some specific naming convention. I don't know how this will go. My friend told me to code this in NodeJS, because then I can port it easily into mobile application. I can do JavaScript, but I prefer Python syntax. Is he right or should I continue in Python?

>>1656 >My friend told me to code this in NodeJS, because then I can port it easily into mobile application. I am totally unfamiliar with mobile programming, but do you have the same sort of access to the filesystem in a mobile application as you do in a desktop one? I thought the OS filesystem was heavily sandboxed per-application, which would make a program like this either useless, or need significant changes anyway.

Who a learn to make a script? I have so many ideas but i don't know how to start it. I start to programming fews days ago.

>>3881 https://docs.python.org/3/tutorial/index.html https://docs.python.org/3/tutorial/modules.html#executing-modules-as-scripts

Wrote a simple script to delete duplicate files in a directory. Any feedback or improvements will be appreciated Python #! /usr/bin/env python3 '''delete duplicate files''' import os import filecmp import itertools while [x for x in map(os.remove, [ x[0] for x in ( [filename for filename in it] for filesize, it in itertools.groupby( sorted( os.listdir(), key=os.path.getsize ), os.path.getsize ) ) if len(x) > 1 and filecmp.cmp(*x[:2]) ] )]: print('iterating')

>>4402 its ugly unreadable soykaf(Elaborate on your opinions; avoid spilling soykaf.)

>>4402 >feedback or improvements will be appreciated - Use a "__main__" test and main() method for executable scripts. https://docs.python.org/3/tutorial/modules.html#executing-modules-as-scripts - The file sizes are read once for sorting and again for grouping. This is wasted work, even if the implementation caches the results for you. What you should do is decorate the items with their size, then sort and group with an itemgetter from operator, then optionally undecorate. Then move this logic out to a sort_and_group method that takes a data source and a property reader. - [filename for filename in it] is pointless verbosity for list(it). - There is no need to make the map target a list. A generator will suffice, like the one that wraps the groups. - Hardcoding os.remove into the duplicate processing logic is bad form. You might want to list the duplicates instead. The duplicate handler should come in as a parameter. - [x for x in map(...)] is simply list(map(...)). - By relying on a while and handling only x[0] you are throwing away all the rest of the listing, sorting and grouping work within each group, only to redo virtually all of it on the next pass. - There is no guarantee that two matching files will end up as neighbors after sorting by size. Consider size(f1) == size(f2) == size(f3) with f1 matching f3 but not f2. Your script only works reliably if the duplicate files happen to be the only ones within their size group, it doesn't correctly remove duplicates in the general case.

>>4404 You should have seen how ugly it was as a one-liner >>4407 Thanks! > list() I totally forgot I can just expand iterators with list(). > __main__ I know it's bad practice, I just dumped it in a file so I wouldn't forget it > decorate items with their size What, like a dict of keys (name, size)? > hardcoding os.remove I wanted to write something self-consuming > statistics yeah that's right. I can just use itertools.combinations and pick out pairs for which filecmp returns true. I also had a slight assumption (which will have failed eventually) that there wouldn't be plenty of duplicates in each group where I used this script. Just wanted to clean directories where I used thread-downloading scripts to get all the images

>>4402 It feels very backwards. Why is all the logic in the while clause? Why are you using map to apply a function with no return value? I know writing programs in a sort of point-free style is nice and all, but Python just doesn't feel like the right language to do it.

>>4412 > It feels very backwards. It could be. I wrote the comprehension first, then needed to think of a way to repeat it > Why is all the logic in the while clause? to make it repeat. each iteration the generated list will be shorted, until you end up with an empty list, which evaluates to False and the while terminates. > Why are you using map to apply a function with no return value? Because I don't care about the return value but I want to map it to the list. It returns None, which guarantees the list won't be empty > Python just doesn't feel like the right language to do it. Could be. I was dicking about in the REPL trying to figure out a solution. This entire ugly mess was born as a one-liner

>>4410 >What, like a dict of keys (name, size)? A plain tuple will do and is cheaper. The decorating technique works like this: >>> import itertools >>> import operator >>> def sort_and_group (source, property): ... decorated = map (lambda x: (x, property (x)), source) ... get1 = operator.itemgetter (1) ... sg = itertools.groupby (sorted (decorated, key=get1), key=get1) ... get0 = operator.itemgetter (0) ... undecorated = itertools.starmap (lambda k, g: (k, map (get0, g)), sg) ... return undecorated ... >>> def gbtolist (gb): ... return list (itertools.starmap (lambda k, g: (k, list (g)), gb)) ... >>> data = range (42) >>> prop = lambda x: x % 7 >>> gbtolist (sort_and_group (data, prop)) [(0, [0, 7, 14, 21, 28, 35]), (1, [1, 8, 15, 22, 29, 36]), (2, [2, 9, 16, 23, 30, 37]), (3, [3, 10, 17, 24, 31, 38]), (4, [4, 11, 18, 25, 32, 39]), (5, [5, 12, 19, 26, 33, 40]), (6, [6, 13, 20, 27, 34, 41])] >>> def countem (f): ... def g (* args): ... g.count += 1 ... return f (* args) ... g.count = 0 ... return g ... >>> prop = countem (prop) >>> gbtolist (sort_and_group (data, prop)) [(0, [0, 7, 14, 21, 28, 35]), (1, [1, 8, 15, 22, 29, 36]), (2, [2, 9, 16, 23, 30, 37]), (3, [3, 10, 17, 24, 31, 38]), (4, [4, 11, 18, 25, 32, 39]), (5, [5, 12, 19, 26, 33, 40]), (6, [6, 13, 20, 27, 34, 41])] >>> prop.count 42 >>> The advantage with expensive property functions is that they will be called only once per item instead of twice.

>>4415 >I don't care about the return value but I want to map it to the list. It returns None, which guarantees the list won't be empty You can apply a None-returner to every item by doing any(map(...)). When you need a count you can do sum(map(lambda x: 1, map(...))). Both of these avoid uselessly storing the entire list of Nones.

I want to focus my time on learning a scripting language. I learnt Python3 in a first year CS course. It was fun, but after learning C the idiosyncrasies of Python frustrate me. I'm thinking about Ruby instead. Do you guys prefer Python to Ruby and why? Should I learn to like Python's characteristics?

>>4418 But I need the list of Nones for when it gets empty I know I consumed all the duplicates (theoretically, anyway, another anon showed the assumptions don't always hold)

>>4420 >But I need the list of Nones for when it gets empty No, you don't. The second method from >>4418 gives you the element count. I'm sure you'll be able to decide emptiness from the element count, somehow.

>>4419 Python is full of idiosyncrasies and edge cases, to me it feels like a huge hack overall, written without a good overall vision of what the language should be about. Ruby, on the other hand, is much more well thought out and offers you much more sophisticated means of abstraction, which are used to implement much of the language. I don't even think the two languages are very comparable; Ruby is just much more powerful, and it makes great use out of its interactive runtime, much like a Lisp, while Python feels like it could just be turned into a compiled language with a minimal runtime without losing much. I recommend Ruby. It makes sense.

>>4421 > please be patient I have autism I probably won't have time to try it until the weekend but I think I catch your drift

>>4424 Not that guy, but I suggest you find out the things that are being reused throughout the loops and most importantly reorder the loops not to be so nested. This is why I think learning loops through recursion is so useful: the values that change and those that are being reused in each loop become very apparent.

I've been using python3 for some django stuff recently and have been interested in using it for cryptography as well. If anyone's interested, check out "cryptopals" which has sets of challenges you can complete in any language. I'm working on challenge 1.6 right now and have been having a blast so far. From looking at other py3 code and coming from Go, it's intimidating how terse people can make their statements! I'm still mucking around in for loops and am using the aforementioned challenges to learn useful bits like zipping, generators, and neat one-line iterators.

>>59 Seconded. Are there any recommended books for machine learning with python 3?

>>4629 >>59 see >>3681

How different is Python 3? If im comfortable with python 2 can I python 3 no problem with little headache? Why has python 2 -python 3 been such a conundrum ?

>>4639 https://docs.python.org/3/whatsnew/3.0.html

>>4639 Pythonistas dont seem to like to talk about this v2 vs v3 issue so I will. I think the most major non-backward compatible change is switching to unicode strings. I think the reason they dont like talking about it is that most the changes seem to make Python more Ruby-like, like by having a common iterator and comparison interface between container types and having Ruby bracket string formatting. Guido should have been more forceful about deprecating v2 so he is at fault by allowing everyone to keep using it, stupidly thinking they would slowly switch on their own.

>>4643 https://learnpythonthehardway.org/book/no

>>4644 linkynoworky

>>4639 Python is very popular among people who don't like programming and just want to get something working without thinking too hard. So the majority of Python users don't really appreciate how Python2 is flawed and how Python3 attempts to fix things. So the critical mass of users have ignored Python3 and stayed with Python2. This Python-for-beginners meme is pretty cruel. It's like teaching someone who wants to be a chef how to use a microwave.

>>4647 I may be dating myself, but I remember a time when Perl was recommended to newbies; it being a dynamic scripting language with libraries for everything. Is Python uniquely ill-suited in this regard? Or is it that there isn't such a thing as a beginner language?

>>4649 I don't think it's a bad language to teach to beginners, the amount of libraries and help online will help just anybody get up to speed and get something done quickly, especially if they don't have somebody tutoring them, and stuff like type systems is just a lot of mental overhead when you still have to learn about basic concepts like functions, control flow and data structures. I just wish those beginners would step outside their comfort zone and try something else at some point, but it's hard to teach people to behave like adults.

>>4650 I get your point. Yes, indeed. One could go do Python exclusively when learning, land a job writing it and then that could be it. It's hard to say that a language being so damn convenient is a drawback, but you do have a point. While moving from one European country to the next, the thing on my resume that would consistently land me jobs was my Python experience, despite my having more experience in other mainstream languages. A lot of neat things can't be done or learned by using it, just because of its peculiarities, and I do think the dev community is poorer for it. Just like learning Latin can make you better off, even if you won't really use it in everyday life, dipping your toes in C-like langs (or other exotic languages) can leave you better off as a developer. But without any outside (on inside pressure), we'll keep having monolinguists (hello JS).

>>4649 >Is Python uniquely ill-suited in this regard? Or is it that there isn't such a thing as a beginner language? I was going to say what >>4650 said. People who learn Python first tend to get stuck with Python and stop growing without realizing it. Python as a language is very sloppy and unsafe. It is optimized for quickly solving typical, predictable problems with the "there is only one way to do things" mantra. But just because your code solves the immediate problem doesn't mean it is robust or secure or maintainable or scalable or "good" for any definition of the word. Beginners get hooked on the instant gratification, "oh look it works after 5 seconds of typing, lets leave it and do something else hurr durr". I've found that looking through open source projects, anything written in Python has a high chance of being utter garbage because beginners only learn how to get stuff working, not how to engineer a system correctly. I think Python shares a lot of the blame because as a language and ecosystem, it focuses very heavily on just getting things working quickly and ignores all the other aspects required for building high quality software. So yeah, people who don't care about writing software per se like physicists and pentesters, Python is the way to go. But teaching it to budding software engineers is a mistake.

>>4644 >>4646 >>4647 This man said this: https://learnpythonthehardway.org/book/nopython3.html I started on Python 2, then learned some C, enough to appreciate the conveniences of Python. Then I learned R and I've mostly been working with Python and R. Is there a language thats very similar to Python with static typing I wonder?

Does anyone have a Introduction to Machine Learning with Python pdf?

>>13 Where the fuarrrk did this logo come form? Python 3 uses the same logo as python 2

We also need a Python Hate thread

>>5673 We don't need one, this is stupid. The design flaws of Python are very well known. If those flaws are a concern for you in general or for a particular problem, just don't use Python.

>>4654 >Is there a language thats very similar to Python with static typing I wonder? Scala has a good static typing system with type inference. There is a lot of effort going on in creating a native compiler and moving away from the JVM, if writing programs for the JVM is a concern.

How the fugg do I remove deleted object from ORM's? Deletion from the actual DB is successful, when it still remains in the Session. I have tried to do `session.expunge(original_object)` - exits with an error, if this line goes after `session.commit()` or does nothing, if is put before. Running tests with pytest. # ------ db.sql ------ CREATE TABLE TestTable ( test_table_id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, time DECIMAL(15,3) NOT NULL DEFAULT UNIX_TIMESTAMP(CURRENT_TIMESTAMP(3)), value VARCHAR(1) NOT NULL, tx VARCHAR(5) NOT NULL, CONSTRAINT pk_TestTable PRIMARY KEY (test_table_id) ); # ------ test.py ------ from sqlalchemy import create_engine from sqlalchemy import Column, BigInteger, Numeric, String, text from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker Base = declarative_base() metadata = Base.metadata engine = create_engine('mysql+pymysql://test:test@localhost/test') Session = sessionmaker(bind=engine) session = Session() class TestTable(Base): __tablename__ = 'TestTable' test_table_id = Column(BigInteger, primary_key=True) time = Column(Numeric(15, 3), nullable=False, server_default=text('UNIX_TIMESTAMP(CURRENT_TIMESTAMP(3)')) value = Column(String(1), nullable=False) tx = Column(String(5), nullable=False) def __repr__(self): return "<TestTable (id='{0}', time='{1}', value='{2}', tx={3})>".\ format(self.test_table_id, self.time, self.value, self.tx) _object = {'value': 'a', 'tx': 'txin'} def new_object(**_object): object_1 = TestTable(**_object) session.add(object_1) session.commit() return object_1 def object_to_delete(object_1): # Create another entry based of original object_2 = TestTable(value=object_1.value, tx='txout') session.add(object_2) # Delete original entry session.delete(object_1) return object_2 def test_sql(): session.query(TestTable).delete() original_object = new_object(**_object) another_object = object_to_delete(original_object) session.commit() # session.expunge(original_object) # """Instance <TestTable at 0xFF> is not present in this Session""" # Fetch the only entry in DB - `another_object` assert len(session.query(TestTable).all()) == 1 assert session.query(TestTable).first() == another_object # Fail assert original_object == None

>>5784 >it still remains in the Session What supports this conclusion? >assert original_object == None All this checks is the value of a local variable of the test_sql invocation. The last thing that happened to it was receiving the value of the new_object call. The session will not magically reach into your stack frame and assign None to one of your local variables.

>>5788 Sounds legit. del original_object and, basically, abandon this practice is the way to go then?

>>5810 There's no real reason to delete original_object in this case, since test_sql is about to finish and its stack frame will go away, along with all its local variables. If you had a long-lived method and a large object that you didn't need anymore, then either deleting the local variable or assigning a sentinel value like None to it would do the trick, since it would reduce the object's reference count by one and collect it if it reached zero.

Does anyone have examples of multi-threading program with more than 2 threads which share single event lock? Or in case I have 3+ threads, I should not use event?

>>5819 >event lock This is a confusion of terms in the context of Python. You either have an Event [1] or a Lock [2]. >3+ threads The mere fact that you have 3+ threads does not impact on the suitability of either Events or Locks, provided they were otherwise appropriate in your scenario to begin with. Keep in mind that Events will allow all waiters to proceed once they wake, and they are not context managers, whereas Locks are context managers and any awakened waiter has to hold the lock before proceeding. [1] https://docs.python.org/3/library/threading.html#event-objects [2] https://docs.python.org/3/library/threading.html#lock-objects

>>5823 What I mean, I have three threads like this: #Thread-1 while True: if something: one_event.set() if something_else: another_event.set() #Thread-2 one_event.wait() do_stuff() one_event.clear() #Thread-3 another_event.wait() do_another_stuff() another_event.clear() It does not matter, which Thread acquires lock, only it would be able to run then? Also, what if I have to run a Thread-4 disregarding the Lock state? #Thread-4 while True: #runs no matter what do_stuff_loop()

>>5842 >acquires lock >the Lock state The code you've shown doesn't contain any Lock objects. Use of set() and clear() implies you have Event objects. The terms are not interchangeable, see >>5823. >It does not matter, which Thread acquires lock There is no acquiring because you don't have locks. >only it would be able to run then? No. Assuming one_event and another_event are not the same object, Thread-2 and Thread-3 can proceed at the same time if Thread-1 calls set() on both events. If on the other hand one_event and another_event are the same object, then a single set() call by Thread-1 is enough to allow both Thread-2 and Thread-3 to proceed at the same time. Also: - Thread-1 will keep spamming set() calls even though Thread-2 and Thread-3 only wait once. - The events are cleared without synchronization with Thread-1. - The access to the 'something' and 'something_else' variables is likely to require synchronization. >what if I have to run a Thread-4 A thread only requires synchronization if it has to deal with shared mutable state.

>>5847 What I am showing you below is a sane approach or Indian-tier? from threading import Thread, Event import time event_odd = Event() event_even = Event() class ControlThread(Thread): def __init__(self, event_odd, event_even): Thread.__init__(self) self.event_even = event_even self.event_odd = event_odd def run(self): i = 0 while True: time.sleep(3) i += 1 if i%2 == 0: self.event_even.set() continue else: self.event_odd.set() continue class Odd(Thread): def __init__(self, event_odd): Thread.__init__(self) self.event_odd = event_odd def run(self): while True: self.event_odd.wait() print('Odd') self.event_odd.clear() class Even(Thread): def __init__(self, event_even): Thread.__init__(self) self.event_even = event_even def run(self): while True: self.event_even.wait() print('Even') self.event_even.clear() class Whatever(Thread): def __init__(self): Thread.__init__(self) def run(self): while True: time.sleep(1) print('Python is like kicking dead whales down the beach') if __name__ == '__main__': t1 = ControlThread(event_odd, event_even) t2 = Odd(event_odd) t3 = Even(event_even) t4 = Whatever() t1.start() t2.start() t3.start() t4.start() t1.join() t2.join() t3.join() t4.join()

>>5853 Event objects are not the appropriate synchronization primitive for this scenario. Consider for example Odd still being in the print call when ControlThread calls event_odd.set() again, then Odd clearing the event and waiting for yet another one, thus skipping an event entirely. What ControlThread effectively does is hand out permissions for Odd and Even to do their thing, while these two consume the permissions they receive from ControlThread. When you have a system that produces and consumes permissions in this fashion, the appropriate synchronization primitive is the Semaphore. Here you would use one for Odd and one for Even. This way several permissions could build up without being lost, and several permissions could be consumed in a row without going into debt. Additionally, while printing is likely to work correctly in this case, the docs do not guarantee that print will be thread-safe in the general case and across all implementations. Since the output stream is a shared resource, it would be prudent to ensure mutual exclusion among printing operations, by having all threads that print acquire a single Lock object for the duration of the print call. And using 'continue' just before the 'while' is about to loop anyway is pointless. [1] https://docs.python.org/3/library/threading.html#semaphore-objects

I was trying to implement the following architecture, however nothing comes to my mind of how it should actually be done. Could anyone transform this pseudocode into a working python piece? Main: i = 0 while True: i += 1 if i%2 == 0: start Even continue else: start Odd continue Even: while True: # No race condition, so when Even acquires control, # Main has no chance to increment `i` thus calling `Odd` prevent Main from running print('Even') time.sleep(5) give control to Main Odd: while True: # No race condition, so when Odd acquires control, # Main has no chance to increment `i` thus calling `Even` prevent Main from running print('Odd') time.sleep(3) give control to Main Whatever: while True: print('Whatever') time.sleep(1) I was looking up the docs each time and also should say, that I have finished a book called "Python Parallel Programming Cookbook" (it's not that good though, a lot of errata).

>>5913 There's a few things that could be done here. First of all, since work done by Even and Odd is going to essentially be serialized, and since Main doesn't do any meaningful work other than controlling Even and Odd, you could accomplish the same thing by having Even and Odd being methods and just calling them in a loop without any concurrency (and then just running Whatever in a separate thread since you don't care about synchronization with it). But obviously you want an exercise in concurrent programming. Let's modify your scenario a bit to make it more realistic. Why would you need Even and Odd to be synchronized? Because they realy on some shared state. Ok, let's say that i is global, shared by all threads. But if that's all that's going on, you could still just serialize this with a bit of mutual recursion. We need a second thread that can run in parallel to Even/Odd, but at times needs to synchronize with them. Let's say that Main, as in your scenario, is responsible for modifying i, but is also responsible for some other work that modifies state that Even/Odd do not depend on. Now we actually need synchronization tools! Here's the pseudo code for the scenario I'm describing: Main: do_work() wait until we can modify the global i i += 1 if i is even: tell Even it can start working else: tell Odd it can start working

>>5913 >>5917 And here's how I would implement that with Events: http://ix.io/DvX/python. Is this something like what you wanted?

>>5913 Since you now want strict serialization between Main/Even/Odd, this can be modeled with three Event objects: main_may_proceed, even_may_proceed and odd_may_proceed. Each of Main/Even/Odd should wait on their event, clear it, do their thing, then set the event of the next-to-run. Main's event should be initially set in order to allow the machinery to start, and you still have to synchronize all print calls on a single lock because of the 'Whatever' thread.

This >>5924 is correct in the absence of the 'Whatever' thread, though there's no real reason to clear virgin events, and even and odd do not need 'global i' declarations.

Is there any way to compile a python 3.6 program to an executable?

>>6164 There are a couple ways for windows, probably one for linux too but I haven't heard of one

i'm making a course in discrete structures and fuarrrk man, I'm not that experienced in programming as much as i'd like to be. I decided to pickup and read up on some books on python. I know C++ fair enough and I dislike java. It doesn't matter what languages I use for my labs. What do you guys recommend for a programming enviroment? I'm interested in emacs but I heard it had a difficult learning curve.

Does anyone have an example of how to create a safe encrypted tunnel to a network service? I assume it is most easily done with '''Paramiko''', but I am fool (not quite, but still) and completely lost in their docs/examples. Just a simple thing - I have server and client. Client uses TCP to communicate with the help of JSON. I just need server to validate that the client is indeed the client it claims to be, and also this tunnel should obviously be encrypted. Links or even working code would be highly appreciated.

>>6269 You can do something like CRAM (challenge response auth mechanism). Suppose the password is "password". Client opens a connection to the server. Server sends back a randomly generated string, like "\0XEH\MA}%". Client concatenates the password with the string, like "\0XEH\MA}%password", computes the md5 (362cafbdee81b55b2f5d9883fb08fff0) and sends it back to the server. The server computes the same thing, and if they match, then the server knows that the client had the correct password. If you had just relied on the password in plaintext, or the md5 of the password, someone sniffing your traffic could record it and reuse it to impersonate you.

>>6274 This does not solve the problem of encrypted channel, what is your reading comprehension? Paramiko docs are fuсking awful, OOP hell, and /demos/ are no better. I don't want to handle all the math-stuff of encryption by myself, fuсking ssh should do it for me, or else what's the purpose of it had been created!

>>615 Cronjobs. This guy knows where it's at.

>>6276 Chill dude, I just read "verify client identity" and hopped on that. Once you know both sides have the password, you can use that to encrypt a session key and encrypt the rest of the channel with the session key. I just assumed you knew that.

>>6269 >Just a simple thing - I have server and client. Client uses TCP to communicate with the help of JSON. I just need server to validate that the client is indeed the client it claims to be, and also this tunnel should obviously be encrypted. Server listens on localhost and the client forwards the port over ssh. That's the simplest solution. If you came here to ask someone to write your code for your then you will be disappointed.

>>6289 >>6286 Whatever. Finally made it. Library has awful docs and examples, and one is not supposed to read the source on a wrapper of level that high. Fuсk OOP, fuсk bad documentation.

>>6299 >Fuсk OOP, fuсk bad documentation. >>> import unicodedata >>> print ('\n'.join (map (lambda c: "{} U+{:04X} {}".format (c, ord (c), unicodedata.name (c)), "Fuсk OOP, fuсk bad documentation."))) F U+0046 LATIN CAPITAL LETTER F u U+0075 LATIN SMALL LETTER U с U+0441 CYRILLIC SMALL LETTER ES k U+006B LATIN SMALL LETTER K U+0020 SPACE O U+004F LATIN CAPITAL LETTER O O U+004F LATIN CAPITAL LETTER O P U+0050 LATIN CAPITAL LETTER P , U+002C COMMA U+0020 SPACE f U+0066 LATIN SMALL LETTER F u U+0075 LATIN SMALL LETTER U с U+0441 CYRILLIC SMALL LETTER ES k U+006B LATIN SMALL LETTER K U+0020 SPACE b U+0062 LATIN SMALL LETTER B a U+0061 LATIN SMALL LETTER A d U+0064 LATIN SMALL LETTER D U+0020 SPACE d U+0064 LATIN SMALL LETTER D o U+006F LATIN SMALL LETTER O c U+0063 LATIN SMALL LETTER C u U+0075 LATIN SMALL LETTER U m U+006D LATIN SMALL LETTER M e U+0065 LATIN SMALL LETTER E n U+006E LATIN SMALL LETTER N t U+0074 LATIN SMALL LETTER T a U+0061 LATIN SMALL LETTER A t U+0074 LATIN SMALL LETTER T i U+0069 LATIN SMALL LETTER I o U+006F LATIN SMALL LETTER O n U+006E LATIN SMALL LETTER N . U+002E FULL STOP >Cyrillic Small Letter Es U+0441 >bypassing the word filter

>>6213 I use sublime with some add-ons. It's pretty comfy to be honest

Looks like soykaf: a_list = [{'a': 1}, {'b': 2}, {'a': 4}, {'c': 2}, {'b': 4}] a_dict = {} for each in a_list: key = ''.join(list(each.keys())) if key not in a_dict: a_dict[key] = each[key] else: a_dict[key] += each[key] How to do the same via a dict comprehension?

>>6811 >key = ''.join(list(each.keys())) Your frankenjoin uses a superfluous intermediate list and assumes the key is a string. To extract a single-entry dict you can use: >>> d = {'a': 1} >>> k, v = next (iter (d.items ())) >>> print (k, v) a 1 The sane way to have the source data come in would be as tuples rather than single-entry dicts. If you must have dicts you can map a converter over them which also handles multiple entries: >>> import itertools >>> import operator >>> l = [{'a': 1}, {'b': 2}, {'a': 4}, {'c': 2}, {'b': 4}] >>> src = itertools.chain.from_iterable (map (operator.methodcaller ("items"), l)) >>> list (src) [('a', 1), ('b', 2), ('a', 4), ('c', 2), ('b', 4)] A dict comprehension would have to do more work than a single linear pass over the source, because it doesn't have a way to refer to itself while being constructed. Use a reduction instead: >>> import functools >>> def combine (tuples, monoid_op, identity): ... def reducer (accum, item): ... k, v = item ... accum [k] = monoid_op (accum.get (k, identity), v) ... return accum ... return functools.reduce (reducer, tuples, {}) ... >>> def sum_dicts (dicts): ... return combine (itertools.chain.from_iterable (map (operator.methodcaller ("items"), dicts)), operator.add, 0) ... >>> sum_dicts (l) {'b': 6, 'c': 2, 'a': 5}

>>57 using cron will also ensure that if something random af happens and the script crashes, at least 10 minutes later it will run again. if it's just sleeping for 10 minutes and crashes, then how will it start again?

>>6822 Thanks for writing soykaf, but my question was precisely "how to do the same via a dict comprehension". fuarrrking hate it when the question is "How do I do X with Y" and the response is "Y is soykaf, here's how you do it with Z, now brb to my reddit feed and soylent foods".

>>6826 make a script that does not crash

Is there a way to set interpreter-wide time (for tests) without affecting system time? The only thing I was able to find is https://stackoverflow.com/questions/12081310/python-module-to-change-system-date-and-time

I'm new to python programming, just a couple hours in so far. What am I doing wrong here? I keep getting an error and cannot figure it out. import random gold = 0 print("welcome to the world of heroes. here be dragons.") name = input("Enter thy name, adventurer!") print(name + " it is then! TO ADVENTURE!") print("You are standing at a fork in the road with three paths to go.") print("Which one do you pick? [L]eft, [R]right or [M]iddle?") roadChoice = input() def forkInRoad(choice): global gold M = int(1) L = int(2) R = int(3) if choice is random.randint(1,3): print("OH soykaf, A MONSTER APPEARS") else: print("You survived. Have some gold.") gold = gold + random.randint(1,5) print("You now have" + str(gold) + "gold." forkInRoad(roadChoice) python3 game.py File "game.py", line 23 forkInRoad(roadChoice) ^ SyntaxError: invalid syntax

>>7020 From what I can see, you haven't properly spaced your function definition of forkInRoad. This is my first time writing code on lainchan, so bear with me if I format this wrong. To define a function in python, you need to space the definition uniformly. IE: def myFunction(input): print(str(input)) # notice how all these lines a spaced the same # I don't think comments don't need to follow spacing rules, but it looks nicer when they do print("wow this is fun") myFunction("python is so cool") Because you haven't spaced the body of forkInRoad properly, forkInRoad is not recognized as a function that you can call, leading to a syntax error on the forkInRoad() call you are making. I think, could be totally wrong.

>>7021 I just copypastad the code from the editor with code /code tags. Here is a screencap.

>>7023 Got it. You didn't close the parenthesis on line 24 print statement.

>>7025 Ah... Thank you.

>>7023 >>7025 >>7026 Also, don't forget to insert spaces around variables in strings. IE: "You now have" + str(gold) + "gold." with gold = 6 will print as [You now have6gold.] "You now have " + str(gold) + " gold." will fix this as [You now have 6 gold.] This problem crops up earlier with name input / printing. Before you proceed, note that I am not a python wizard and my advice may be patently bad. I'm not quite sure if your if-statement on choice works as intended. I think it is comparing the input (as a string?) to the output of random.randint(1,3). If you want to tie player choice to a set outcome, a dict of outcome functions might work better. Something like: def leftPath(): print("Left") def rightPath(): print("Right") def middlePath(): print("Middle") choiceOutcomes = {"L":leftPath, "R":rightPath, "M":middlePath} # Get the entry from the dict that matches the user choice, then call it as a function # The dict will return an instance of the function # IE: Given "L" you will get an instance of leftPath. # I think, maybe it is a reference to leftPath, this is where my deep knowledge of python is severly lacking. choiceOutcomes[input()]() Just a suggestion, hope it helps, and that I haven't said anything too incorrect.

>>7020 >>7023 >roadChoice = input() roadChoice is a string >if choice is random.randint(1,3): so it will never equal or be identical with an int. You can use random.choice ("LMR") and == instead of 'is'. You can validate the choice with if choice not in "LMR": >print("You now have" + str(gold) + "gold." You have already been told about closing your parens, and print can do spacing and conversion for you: print ("You now have", gold, "gold.")

>>7030 >print ("You now have", gold, "gold.") I've been using python for ages now and somehow never learned this. This is so cool, implicit conversions and all! Thank you lain!

>>7030 >>7034 >random.choice ("LMR") Also, never knew random.choice() could treat a string as a list of options, what a neat idea. Thank you a second time lain!

>>13 Is python any good for writing an AI?

>>7378 It is a very good choice for that and for machine learning in general, because of its ecosystem of numerical and machine learning libraries. However, if you have to ask this question, chances are that the reasons that make it a good choice won't really apply to you in the near term. It'll still be a good choice, but mainly because it is a simple language for beginners and many languages can do that well.

>>7379 I'm experimenting with AI weaponry, on a very small scale.

Relative newbie here. https://pastebin.com/8d5Fncmk I wrote a basic C code generator in Python! I'm planning to add a bit more functionality to it eventually in order to make it more useful. I'm planning to use it for a DSL to write fast custom numpy functions.

>>7463 >Relative newbie here. - In printGenerator, you can print in one go with print ('\n'.join (gen)) for a speedup at the cost of negligible extra memory use. - In indentcodelines, you recompute indentlevel*tabstop at each step, instead of only when indentlevel changes. Consider storing the product in a variable and only updating when needed. - yield from some_iterator can be used instead of a plain 'for' with a 'yield' inside, like in generateCProgram. - Consider using string formatting or f-strings instead of manually adding strings together, because the latter will build each intermediate result as a separate string while the former will use a modifiable internal buffer, thus improving space consumption from quadratic to linear. - In CSymbol symbolTable is a class attribute, yet in __init__ you access it as if it were an instance attribute, which is poor form. Use CSymbol.symbolTable and consider only retrieving it once instead of multiple times. You also have a NameError with the naked symbolTable in the str call, because you haven't tested that code path. - In CVariable initializationStatement only performs declaration. - Use str (FOO) instead of FOO.__str__() . - Use ' ,'.join instead of manually looping to insert separators. - In yieldStatement of IfStatement there's no condexpr. __init__ has the reverse problem. Same with WhileStatement. Write some minimal tests. - You need to add .h in headerInclude.

>>7463 Interesting, but I guess that approach will have very limited application. It will be hard to describe more complicated C-files, that contains, for example, few hundreds loc and custom logic that defined in Python code. I've already heard to work with C-code generation from Python while working on customization of Beremiz IDE (https://beremiz.org/) for my needs. Beremiz is FOSS for automation which uses codegeneration routines to create C files depends of user input in wxPython UI. Then created C-sources compiles and links with cross-toolchain to runs on target PLC platform. And in Beremiz we use plain Python string formatting for this. For example, in CANOpen module we just create template file like this: https://github.com/jubnzv/beremiz/blob/master/canfestival/cf_runtime.c, then call required code generation routines from controller class (you can find more complicated platform-depended examples in targets directory). By this way we need to describe custom codegeneration classes, that, depending on input data, generates required string to substitute them. It sounds easy, but in real project, it is so tediously to modify small part of C logic: need to change few generated loc usually requires completely rewrite most of python codegenerator logic. I was looking for any other solutions, e.g. various existing template and DSL based generators like cog, but all of them looks more complicated and/or not enough functional. So, if you have any ideas about more elegant and universal implementation, you are welcome.

>>7467 Thanks for the thorough review! I'm still relatively new to Python. Most of my experience is with static languages where the compiler would catch stuff like referring to a nonexistent variable, so Python is a bit of a change in workflow for me. Most of the bugs I've encountered so far have been type errors or accessing nonexistent variables, so I'm clearly being sloppy regarding these basic bugs. >>7474 My goal is something rather simple and not particularly big. If the project balloons to more than 2k-ish lines, I'd consider my implementation to be a failure. Basically, I want to write a Python DSL for iterator/stream transformations which are expressed in an itertools-like way and operate with numpy arrays as inputs and outputs. The difference in use would be that instead of chaining the iterators on an input, I would be composing the transformations, kind of like Clojure's transducers. In order to be fast on numpy arrays, the transformation would be compiled to a C function with all loops fused to the greatest extent possible, hence the code generator.

"Forward error-propagation" prototype code. Simple as hell to decode (just do a rolling XOR over key-sized blocks, in this rudimentary implementation), All the data MUST be present to reconstitute the original file, and the magic is that the encoded data is (to damn near the limit possible) only pure random noise. If you remove any single contiguous 4KiB block, the entirety of the remaining data BECOMES absolutely uncorrelated random data. (It is only not-random-noise when all the data is present together.) This demo does not adequately showcase the ultimate use of this: Which is to have an e.g., 4KiB encryption key "tiled" across a 700MiB CD, where you only have to damage any 4KiB sector to render the entire key unrecoverable. At a data density of 0.90 Gbit/in^2 according to Wikipedia, Wolfram Alpha translates that to about 170KiB per square millimeter. So to nuke the original 4KiB of data, you will have to damage more surface area of a CD than that of the tip of an 0.5mm pencil lead (but the surface area equivalent to the tip of an 0.7mm pencil lead would be more than sufficient.) Basically, any amount of crush or stab onto the surface of the CD (or snapping it) would hopefully blow the key away beyond any hope of even super-forensics to restore it. You don't have to go Full Thermite this way, which is nice; it can be used with regular equipment from random import getrandbits from base64 import encodebytes def xor(b1, b2): # Takes two bytes and returns their XOR # https://stackoverflow.com/a/2612730/1874170 return bytes(a ^ b for a, b in zip(b1, b2)) def getrandbytes(size): # https://stackoverflow.com/a/28131868/1874170 BITS_PER_BYTE = 8 return bytes(getrandbits(BITS_PER_BYTE) for _ in range(size)) def encode(key, chunks, media=bytearray()): chunk_length = len(key) media_size = chunk_length * chunks running_xor = bytes(chunk_length) assert len(media) == 0, "Media stubs not supported yet!" for i in range(len(media), media_size - chunk_length, chunk_length): temp_chunk = getrandbytes(chunk_length) media.extend(temp_chunk) running_xor = xor(temp_chunk, running_xor) media.extend(xor(running_xor, key)) return bytes(media) def decode(media, chunk_length): running_xor = media[0:chunk_length] media_size = len(media) for i in range(chunk_length, media_size, chunk_length): running_xor = xor(media[i:i + chunk_length], running_xor) return running_xor if __name__ == "__main__": key_user = "SwAggin1234" key = key_user.encode() chunks = 64 print("Original key is: {}\n".format(key.decode())) encoded_key = encode(key, chunks) print("Encoded key (displayed in base64) is:\n{}\n".format( encodebytes(encoded_key).decode()) ) recovered_key = decode(encoded_key, len(encoded_key) // chunks) print("Recovered key is: {}\n".format(recovered_key.decode())) Is my thinking on this solid and correct? (Repost because indentation br0ek the first time) Obviously it would be preferable to have some kind of support for leading arbitrary data (so an integrated snappable LUKS CD could be made) Unfortunately it would probably have to be ported to C or ASM (or posix-sh w/busybox?) Other possible ideas: have a very robust "must have at least N CDs from the set to reconstitute the key" (would possibly require reconsiderations/extensions to the algorithm) You could have something which loads up one encryption key if you only boot the first CD, but loads the actual extra-secret encryption key if you hold shift to make it wait for the second CD

>>7971 >for i in range(len(media), media_size - chunk_length, chunk_length) ugh kill me https://repl.it/@JamesEdington/AntiRedundantPrototype1 should probably gist it but am on wind00z box atm and don't feel like acquiring dev env

>>7971 >>7972 >blow the key away beyond any hope of even super-forensics to restore it >Is my thinking on this solid and correct? https://docs.python.org/3/library/random.html """Python uses the Mersenne Twister as the core generator. It produces 53-bit precision floats and has a period of 2**19937-1. The underlying implementation in C is both fast and threadsafe. The Mersenne Twister is one of the most extensively tested random number generators in existence. However, being completely deterministic, it is not suitable for all purposes, and is completely unsuitable for cryptographic purposes.""" https://en.wikipedia.org/wiki/MT_RAND """Is not cryptographically secure, unless the CryptMT variant (discussed below) is used. The reason is that observing a sufficient number of iterations (624 in the case of MT19937, since this is the size of the state vector from which future iterations are produced) allows one to predict all future iterations.""" https://github.com/python/cpython/blob/3.6/Modules/_randommodule.c """A C-program for MT19937, with initialization improved 2002/1/26. [...] #define N 624"""

>>7979 Hmm, you're right, I guess Theoretically given enough data (e.g., an undestroyed ~80% of a 700MiB CD), someone could search the Mercenne space to find and reconstruct the destroyed random data parts (and thus have the information required to reconstitute the key) So, you'd need a proper random data algorithm But the idea is otherwise sound, right?

>>7981 >But the idea is otherwise sound, right? With stringent conditions on the CSPRNG and its seeding, the quick destruction property claimed in >>7971 would hold. However, it also looks like a solution looking for a problem, as well as a waste of entropy. https://en.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator

Where to start with python 3? Some online courses like codeacademy + books? Im new in programming, and i have small problems with math. Better skill up mathematick or keep it and learn python?

>>7986 I enjoyed Dive Into Python 3. It's available for free online. >>8201 again, you do NOT need to use python3 to use tensorflow.

Some 50+ projects for newbies to practice python3. Hope some of you will find this useful! :)

>>7979 There's a cryptographically secure version of MT functions. It was written in a paper by one of the authors of the mersenne twister. I'll look for it but it was a simple patch. If one wants really secure bitstreams you can create a zener electron tunelling bitstream. You can make these simply within an arduino and stream them. They use a very simple quantum effect with zeners and their breakdown voltage. The zener, when the voltage is at threshold will either keep or breakdown at an absolute entropy full manner. There are cheap devices that utilize this method. http://iank.org/trng.html

Trying to implement a (just [24,18] for now) Hamming encoder But it's so hard to find information online that I don't even have some data points to start with a test suite to see if my implementation is even valid #!/usr/bin/env python def __main__(): from sys import argv,getfilesystemencoding code=(24,18) if len(argv) == 1: # bytes([0,192,168]), roughly data = 0b001100000010101000.to_bytes(3, byteorder='big') else: data=argv[1].encode('iso-8859-1', 'surrogateescape') # https://stackoverflow.com/a/27704783/1874170 print('Data in:\t{}'.format(' ' * (code[1]%8) + ' '.join([format(i, "08b") for i in data])[code[1]%8:])) encoded = hamming(data, code) print('Encoded:\t{}'.format(' ' * (code[0]%8) + ' '.join([format(i, "08b") for i in encoded])[code[0]%8:])) def _xor(items): i=0x00 for item in items: i ^= item return i _mask_le = lambda B, i: B & 0b0000001 << i # _index_le(0b0001, 0) == 1 and _index_le(0b1011, 2) == 0 # Negative indices TODO, perhaps infeasible due to non-existant "left" boundary _index_le = lambda B, i: _mask_le(B,i) >> i def hamming(data, code=(24,18)): # For now we only do [24,18] encoding assert code == (24,18) # Ensure the data is of appropriate length assert -(-code[1] // 8) == len(data) # https://stackoverflow.com/a/33300093/1874170 # https://pdc.ro.nu/hamming.html check_mask={ (24,18):[ (1,[17,15,13,11,10, 8, 6, 4, 3, 1, 0]), (1,[17,16,13,12,10, 9, 6, 5, 3, 2, 0]), (1,[17,16,15,14,10, 9, 8, 7, 3, 2, 1]), (1,[10, 9, 8, 7, 6, 5, 4]), (1,[17,16,15,14,13,12,11]), (0,[17,14,12,11,10, 7, 5, 4, 2, 1, 0]) ] } encode_table={ (24,18):[ [( 1, 3),( 0, 3),( 0, 2),( 0, 1),( 1, 2),( 0, 0),( 1, 1),( 1, 0)], [( 1, 4),( 0,10),( 0, 9),( 0, 8),( 0, 7),( 0, 6),( 0, 5),( 0, 4)], [( 1, 5),( 0,17),( 0,16),( 0,15),( 0,14),( 0,13),( 0,12),( 0,11)] ] } chex_int=0x00 data_int=int.from_bytes(data, byteorder='big') I=0 for B in check_mask[code]: chex_int ^= ( _xor( [B[0]] + [_index_le(data_int, i) for i in B[1]] ) << I ) I += 1 encoded=bytearray() bc_int=(data_int,chex_int) I=0 for B in encode_table[code]: x=0x00 i=0 for b in B: x ^= _index_le(bc_int[b[0]], b[1]) << i i += 1 encoded.append(x) I += 1 return bytes(encoded) __name__ == "__main__" and __main__()

>>8237 https://clsr.net/_zed_shaw_is_a_hack

So, I have been hired as a data scientist in a python shop. While I learnt programming in python, the past 2 years I have mainly worked in R. What resources would you recommend to get the rust off my python? The fluent python book in this post seems quite appropriate, and I was also thinking about reading one of those think _____ by allen downey (http://greenteapress.com/wp/) just to read on a topic while I also read some good python. Any other recommendations?

>>8573 From the previous e-book thread. Can't speak to its quality.

>>8403 Eh Zed Shaw has many perfectly valid points. Initial versions of Python 3 were total disasters with errors every where you look. It's mostly people way too invested in python 3 who feel attacked and respond with hate. Hell most people who attacked Zed didn't even get the Turing complete joke he initially had in his article. Not getting that joke immediately means you barely pass as a computer scientist, and in my opinion, if you don't get that joke you don't get to criticize the guy for making perfectly valid points.

>>8592 Most people didn't interpret it as a joke, they interpreted him as being serious. If it were more obvious that it was a joke, it wouldn't have been as bad, but there was no indication that it was a joke in the original version, on the contrary, he included a short explanation of *why* he made the statement which served to only make him appear more incompitent if you were not interpreting it as a joke. Aside from that, anyone trying to handle multiple encodings in python 2 can tell you that python 3 was the right solution. The print() breaking backwards compatibility was probably python3's largest mistake, it really divided the community by making a clear difference between the languages for all programmers, especailly when probably more than half of programmers never deal with unicode encoding.

>>8595 Yeah that's my point. It's an obvious joke. Fact is, most people who use python don't have computer science degrees.

>>8598 Did you even read my post? I said that even if you do have a CS degree and know all about the subject, it still wasn't obvious that it was a joke, the more obvious conclusion was that he's a dumbass. Especially because of the fake explanation about the inability to run py2 over py3

>>1174 Unfortunately this link is not available anymore, would anyone care to upload the file again? Thanks!

Zed Shaw is absolutely a hack and an asshat who's only famous for being very vocal about his bad opinions, and his book is fuarrrking awful too. Please don't recommend it to beginners. That said, most people had pretty good reasons not to switch to Python 3 when it came out: - As previously mentioned, Python 3's first few versions were buggy. - The tool for automatic conversion (2to3) was pretty poor and still required a lot of manual work to convert a big codebase to Python 3, and the code it produced was not compatible with Python 2, which meant that if you wanted to move your codebase to 3 you'd have to fix all the libraries yourself, or wait until the libraries added compatibility for Python 3. - There were not many strong incentives to move to 3 except for the unicode strings, which was not a big concern for every Python user. - Python 3 has been slower than Python 2 until just a few months ago.

>>8403 Zed was right; Py3k was a colossal waste of time. There was zero reason to require a mass rewrite of all existing code to a new language, because that's what Py3 is, a different language than Py2. If it can't run Python2 code, then it isn't Python. Py2 will continue to plague Py3 programmers for decades via legacy projects. A compatibility layer would have been messy, but it would have been the *right* thing to do.

>>8709 To sum up the Python3 developers. Every time you complain Python3 can't run Python2 code, they tell you it's a new language and shouldn't be able to. Every time you say you want to stick with Python2, they say they're the same language and you have to upgrade.

>>8637 It was obviously a joke to me and I didn't have a degree at the time I read it.

>>7380 Please explain how you're doing this. >>59 >>4629 I have a machine learning with python pdf on my laptop. Fair warning: it's not a meme. It has advanced calculus, some neurology, advanced statistics, and of course programming. I'll upload it here tonight if it's not too big

>>8717 I am still mad at google for restricting tensor flow support to several very specific versions of python. WTF google. I feel like python projects in general are getting more and more finicky with dependencies. Why? WHY?

>>8717 Delivered If someone has more books, or something like wargames for machine learning, I'd appreciate it. >>8729 To be honest with you, I'm more of a C++ kind of guy. I like to have full control over programs. Although, for more programming work per hour, Python and its libraries are superior. I feel you that it's annoying having to learn all of the functions of someone else's code, and even more annoying when they change (or disappear).

>>8731 I'm just saying that to install tensor flow at all you have to install a non-standard version of python. That's ass. It means you have to keep a separate environment for their soykaf. What does everyone there use docker or something?

>>8710 The only major difference between Py2 and Py3 is that Py3 introduces unicode support. I cant understand why someone doesnt make a fork of Py2 that supports unicode and then they will be compatible

>>8734 Wrong. They subtly changed a bunch of the apis in the core libraries. I've run into a bunch in code where the stupid developer didn't write in a shebang to let me know it was python3 exclusive. Names of methods change.

So. I am using Python 3.6 and I need a hint on the app's architecture approach. Note, in my post I use "type" not as in "python type" but as a "kind". I have multiple external APIs. I have to unify and integrate those, constructing queries within my app which request same identical actions on these different APIs. - Should I store actions as strings in a DB? - Or make classes for each type? - Or make parent class with some method "bullets" defined (even though the syntax also differs from API to API), then subclasses with `__init__` for their specific verbs, which are then fed into parent class methods with `__super__` ? If this, how do I organize the app, so it knows which APIs to query each time - with a simple `if` logic defined for each case which checks some parameters or what? - Or do I make an abstract class (I have not tried this stuff yet) with all the possible actions defined, then subclass for each API type and override the methods (well, that's basically what I have described previously, but spiced up with OOP-poopery?)? How is this problem called? There must be a buzzword for this, googling which leads to soyboy articles of besterest practices.

>>8637 If you really want me to spell it out for yoy, I'm saying most of the people who criticized zed shaw about python 3 are dumb and emotional.(Argue in good faith.)

>>8782 The idiot who teaches python to other idiots is an idiot? Not surprising for a language designed for idiots.(Argue in good faith.)

help me out, please. I would like to traverse through a directory structure that looks like this: topdir dir1 excel1 excel2 dir2 excel3 etc I am scraping the excel's column B for a string, and that is working nicely. However, my script only goes through the topdir, and doesn't go into the subdirs. Below is my code: #!/usr/bin/python3 '''scan a folder recursively for *export.xlsm, load file scan column B for an error message IF found, list on terminal or print to file ''' import openpyxl import os import sys import warnings warnings.simplefilter("ignore") targetString = str("Error22") scriptPath = os.path.abspath(__file__) scriptDir = os.path.split(scriptPath)[0] topdir = '.' fileExtension = '.xlsm' def scrapeSheets(): for i in os.listdir(os.chdir(scriptDir)): if i.endswith("export.xlsm"): print("Working on:", i) wb = openpyxl.load_workbook(i, data_only=True) sheet = wb["data-sheet"] for cellObj in sheet["B"]: if cellObj.value == targetString: print(cellObj.row, cellObj.value) def mainLoop(): for dirpath, dirnames, files in os.walk(topdir): for name in files: if name.lower().endswith(fileExtension): scrapeSheets() if __name__ == "__main__": mainLoop() As I said, the scraping works, but I cannot go into the subfolders, and I cannot figure out why. I'm new to programming.

>>8847 The problem: - You are mixing traversal responsibilities between mainLoop and scrapeSheets. - scrapeSheets takes no arguments so it knows nothing about the dirpath, dirnames, files and name local variables of mainLoop. - You are walking the current directory but listing the top-level component of the script directory. - The filename test is repeated inconsistently and inline. The solution: - Remove traversal and filtering responsibilities from scrapeSheets and dump chdir. - Move the filename test into its own function. - Give scrapeSheets an argument for the file it is to operate on and start it from the print/load. Drop the plural. - Make up your mind about the current directory versus the script directory. - Use the filename test in mainLoop and call scrapeSheet with os.path.join (dirpath, name) .

>>8849 Thanks, I figured it out with glob, however.

>>14 Ewww. json_data = requests.get(url[:-4]+'json') Just use https://github.com/vichan-devel/vichan-API and make it clear that you're already using it. url[:-4] is just obfuscative.

>>14 An idea I just got is that; instead of sending you an e-mail, you can just send the user a desktop notification using notify-send. So I modified the code to send me a notification whenever there's a new post. Thanks lain!

>>14 >>9331 Seeing i'm a fuarrrking year late... Anyways if there are still any lainons in this thread; i just wanted to mention that i forked >>14's project and made it so that you could run it from your terminal and get notifications using notify-send.

>>9415 https://git.fuwafuwa.moe/fridmanos/python-scripts

>Python 3 >Also discuss and share projects you have been working on. DHALF.TXT >>8821 Nearest color search for error diffusion >>10181 The scanning direction can be modeled by a boolean stream. class ErrorDirections: @ staticmethod def ltr (): return itertools.repeat (True) @ staticmethod def rtl (): return itertools.repeat (False) @ staticmethod def serpentine (): return itertools.cycle ((True, False)) A filter, as far as the error diffusion engine is concerned, declares its maximum dispersion bounds and is able to disperse one error into the error matrix. class ErrorFilter: def rows_below (self): pass def lateral (self): pass def disperse (self, column, error, error_rows, rows_delta, ltr): pass

Using a nearest color search, some scanning directions and a filter, an error diffusion engine can be constructed. It is also responsible for: - building a clamping table to limit the error-corrected color - using widened error rows tailored to the filter to avoid any conditionals when the filter adds errors to the matrix - managing the sliding of error rows by rotating the used up row into the last position class ErrorDither (EachRow): def __init__ (self, nearest, directions, filter): self.nearest = nearest self.directions = directions self.filter = filter self.filter_rows = 1 + filter.rows_below () self.lateral = filter.lateral () self.clamp_table = ErrorDither.build_clamp_table (255, 255) def edit_before (self, image): w = image.width n = 3 * (w + 2 * self.lateral) self.error_rows = [ [0] * n for k in range (self.filter_rows) ] self.zero_row = [0] * n def rotate_and_zero (self): errows = self.error_rows first = errows.pop (0) first [:] = self.zero_row errows.append (first) @ staticmethod def build_clamp_table (below, above): t = [0] * below t.extend (range (256)) t.extend ([255] * above) return t def eachrow (self, row): ltr = next (self.directions) nearest = self.nearest.nearest disperse = self.filter.disperse clamp = self.clamp_table delta = 3 * self.lateral errows = self.error_rows errows0 = errows [0] if ltr: colrange = range (0, len (row), 3) else: colrange = range (len (row) - 3, -3, -3) for k in colrange: deltak = delta + k want = ( clamp [row [k ] + errows0 [deltak ] + 255], clamp [row [k + 1] + errows0 [deltak + 1] + 255], clamp [row [k + 2] + errows0 [deltak + 2] + 255], ) have = nearest (want) # row [k : k + 3] = have row [k ] = have [0] row [k + 1] = have [1] row [k + 2] = have [2] error = ( want [0] - have [0], want [1] - have [1], want [2] - have [2], ) disperse (k, error, errows, delta, ltr) self.rotate_and_zero ()

A filter will precompute error dispersion tables, since there are far fewer error values than pixels. However, dispersion positions and weights do not fully determine the outcome, since there is still a choice in where to place fractional error parts. This choice is given to a distributor, which knows how to distribute one error value. class WeightedDistributor: def set_weights (self, weights, positions): pass def disperse (self, value): pass @ staticmethod def check_weights (weights): if any (w < 1 for w in weights): raise ValueError ("{}: {}".format ( "invalid weights", weights)) With that a filter is just a series of positions and weights, and a distributor for precomputing tables. The actual dispersion called by the pixel loop is just a series of table lookups. class ErrorTableFilter (ErrorFilter): def __init__ (self, weighted_distributor): self.weighted_distributor = weighted_distributor self.posltr = self.positions_ltr () self.posrtl = self.positions_rtl () self.build_tables () def positions_ltr (self): pass def positions_rtl (self): ltr = self.positions_ltr () return [ (-x, y) for x, y in ltr ] def weights (self): pass def rows_below (self): return max (y for x, y in self.posltr) def lateral (self): return max (abs (x) for x, y in self.posltr) def build_tables (self): limit = 255 size = 1 + 2 * limit count = len (self.posltr) data = [ [0] * size for k in range (count) ] wdist = self.weighted_distributor wdist.set_weights (self.weights (), self.posltr) disp = wdist.disperse for v in range (1, limit + 1): values = disp (v) over = v + limit under = limit - v # print (v, '->', values) for pos, dval in enumerate (values): datapos = data [pos] datapos [over ] = dval datapos [under] = -dval self.tables = data def disperse (self, column, error, error_rows, rows_delta, ltr): re = error [0] + 255 ge = error [1] + 255 be = error [2] + 255 pos = self.posltr if ltr else self.posrtl tables = self.tables delumn = column + rows_delta for dxdy, table in zip (pos, tables): dx, dy = dxdy x = delumn + 3 * dx ey = error_rows [dy] ey [x ] += table [re] ey [x + 1] += table [ge] ey [x + 2] += table [be]

One idea for a fractional error part distribution strategy is to sort the positions and weights in some order and assign error parts from the total error in that order. Since this is a precomputation, divs are used for uniformity even when the weights would allow shifting. The total error is used up precisely, as DHALF.TXT advises. class SortingDistributor (WeightedDistributor): def set_weights (self, weights, positions): WeightedDistributor.check_weights (weights) self.sum_weights = sum (weights) self.build_data (weights, positions) def build_data (self, weights, positions): pass def disperse (self, value): left = value div = self.sum_weights data = self.data out = [0] * len (data) for w, k in data: part = left * w // div out [k] = part left -= part div -= w lastk = data [-1][1] out [k] += left return out Two sort strategies are used for now, in ascending and descending order of weights. class DescendingDistributor (SortingDistributor): def build_data (self, weights, positions): self.data = sorted ( itertools.starmap ( lambda k, w: (w, k), enumerate (weights)), reverse = True) class AscendingDistributor (SortingDistributor): def build_data (self, weights, positions): self.data = sorted ( itertools.starmap ( lambda k, w: (w, k), enumerate (weights)), reverse = False)

With all that preparatory work specific filters can be defined by weights and positions: class FlobergFilter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (-1, 1), (0, 1), (1, 1), ] def weights (self): return ( 7, 3, 5, 1, ) class FlobergFakeFilter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (0, 1), (1, 1), ] def weights (self): return ( 3, 3, 2, ) The fancy filters: class JarvisFilter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (2, 0), (-2, 1), (-1, 1), (0, 1), (1, 1), (2, 1), (-2, 2), (-1, 2), (0, 2), (1, 2), (2, 2), ] def weights (self): return ( 7, 5, 3, 5, 7, 5, 3, 1, 3, 5, 3, 1, ) class StuckiFilter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (2, 0), (-2, 1), (-1, 1), (0, 1), (1, 1), (2, 1), (-2, 2), (-1, 2), (0, 2), (1, 2), (2, 2), ] def weights (self): return ( 8, 4, 2, 4, 8, 4, 2, 1, 2, 4, 2, 1, ) class BurkesFilter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (2, 0), (-2, 1), (-1, 1), (0, 1), (1, 1), (2, 1), ] def weights (self): return ( 8, 4, 2, 4, 8, 4, 2, ) And the Sierra filters: class Sierra3Filter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (2, 0), (-2, 1), (-1, 1), (0, 1), (1, 1), (2, 1), (-1, 2), (0, 2), (1, 2), ] def weights (self): return ( 5, 3, 2, 4, 5, 4, 2, 2, 3, 2, ) class Sierra2Filter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (2, 0), (-2, 1), (-1, 1), (0, 1), (1, 1), (2, 1), ] def weights (self): return ( 4, 3, 1, 2, 3, 2, 1, ) class Sierra24AFilter (ErrorTableFilter): def positions_ltr (self): return [ (1, 0), (-1, 1), (0, 1), ] def weights (self): return ( 2, 1, 1, )

Hooking up a nearest color search, a direction stream, a filter and a distributor to the diffusion machinery: @ ParserRegistry.ADD ('derr') class ErrorDitherParser (Parser): DIRECTION_MAP = { 'l': ErrorDirections.ltr, 'r': ErrorDirections.rtl, 's': ErrorDirections.serpentine, } FILTER_MAP = { 'fs' : FlobergFilter, 'notfs' : FlobergFakeFilter, 'jarvis': JarvisFilter, 'stucki': StuckiFilter, 'burkes': BurkesFilter, 'sie3' : Sierra3Filter, 'sie2' : Sierra2Filter, 'sie24a': Sierra24AFilter, } DISTRIBUTOR_MAP = { 'a': AscendingDistributor, 'd': DescendingDistributor, } def editor (self, params): dir, fil, dis = params.split (',') dir = ErrorDitherParser.DIRECTION_MAP [dir] fil = ErrorDitherParser.FILTER_MAP [fil] dis = ErrorDitherParser.DISTRIBUTOR_MAP [dis] return ErrorDither ( # ColorNearestPlain ( # ColorList8 (), # ColorDistance2 ()), ColorNearest8 (), dir (), fil (dis ())) Here's a Lain dithered with error diffusion using serpentine scanning, the descending distributor and the DHALF.TXT filters in exposition order, and the original.

And here's the effect of error diffusion dither on the linear and radial gradients, with the same settings used for the Lain. random dither >>8928 and >>8934 fixed threshold dither >>9363 ordered dither >>9566

>>10330 Could you please post your code and write up see here with proper formatting? I am interested in this .

interactivepython.org has free python books.

>>915 >2:N-1 = 0 >"only points on edges have value 1" >mfw arrays start from 0 cool code. but i can tell from the picture you messed up this part. if you look closely the parts on the left and bottom are at 1 for 2 dots in instead of only at the edges.

So i have this code and I'm trying to model a simple orbit. Any ideas what im doing wrong? ``` import numpy as np import matplotlib.pyplot as plt r0 = [1, 0] v0 = [0, -3] tmax = 10 N = 100 G = 1 M = 1 def orbit_solve(r0, v0, tmax, N, method='euler'): dt = tmax / N r = np.zeros((N,2)) v = np.zeros((N,2)) a = np.zeros((N,2)) G = 1 M = 1 for i in range(0, N-1): r[i+1] = r[i] + v[i]*dt a[i] = -G*M*(r[i-1] / 1) v[i+1] = v[i] + a[i]*dt return r, v ```

>>10464 you seem to be utterly disregarding 3 of your 4 arguments for a start. Also, for the love of all that is holy use a consistent style - you've got caps and lowercase variables all over the place

>>10464 >Any ideas what im doing wrong? - You need to initialize r[0] and v[0] to r0 and v0. - The acceleration update is wrong. By getting i from i-1 instead of i+1 from i like for r and v, the first access is to r[-1], which in python wraps around and gets the last item of r, which is (0,0). You need to compute a[i+1] from the new r[i+1]. Correspondingly, use a[i+1] in the v update. - You also need to compute and divide by the cube of the distance instead of just assuming it will be 1. And you can drop the parens. https://docs.python.org/3/library/math.html#math.hypot - Your G of 1 is suspect. If you are setting M to 1 in order to use the standard astronomical units, then G should be four times the square of pi. - As a minor point, explicitly using a starting value of 0 for a range is superfluous. https://docs.python.org/3/library/stdtypes.html#range

>>10467 I'm following the scientific notations for those letters. >>10471 Thank you! This was very helpful.

Here's one way to generate character stamps in pycairo: def work_char (data): size = data ["size" ] back = data ["background"] char = data ["character" ] fore = data ["foreground"] name = data.get ("name") bold = data.get ("bold", False) surf = cairo.ImageSurface ( cairo.FORMAT_RGB24, size, size) ctx = cairo.Context (surf) ctx.rectangle (0, 0, size, size) ctx.set_source_rgb (* back) ctx.fill () ctx.set_font_size (size) ctx.set_source_rgb (* fore) if bold: font = ("", None, "bold") ctx.select_font_face (* kar.src_font_face (font)) ext = ctx.text_extents (char) xb, yb, w, h, xa, ya = ext x = (size - w) / 2 - xb y = (size - h) / 2 - yb ctx.move_to (x, y) ctx.show_text (char) out = name or "{:04X}".format (ord (char)) pngexport (out, surf) Heart stamps for the corners of the {0,255}^3 cube: class Char8: @ staticmethod def charloop_fixed (ch): return ch * 8 @ staticmethod def fore_shade (n, back): fore = tuple (map ( lambda x: 0.75 * x, back)) if sum (fore) == 0: fore = (0.25, 0.25, 0.25) return fore def main_char8 (argv): size = int (argv [0]) text = Char8.charloop_fixed (chr (0x2665)) forefunc = Char8.fore_shade for k, ch in enumerate (text): back = tuple (map ( lambda b: 1 if k & (1 << (2 - b)) else 0, range (3))) fore = forefunc (k, back) work_char ({ "size" : size, "background": back, "character" : ch, "foreground": fore, "name" : "char{}".format (k), "bold" : False, }) And the font spec: def src_font_face (data): if isinstance (data, str): return (data,) family, slant, weight = data if slant is None: slant = cairo.FONT_SLANT_NORMAL elif isinstance (slant, str): slant = getattr (cairo, "FONT_SLANT_" + slant.upper ()) else: pass if weight is None: weight = cairo.FONT_WEIGHT_NORMAL elif isinstance (weight, str): weight = getattr (cairo, "FONT_WEIGHT_" + weight.upper ()) else: pass return (family, slant, weight)

Here's an image expander that replaces pixels with one of eight equally sized stamps for the corners of the {0,255}^3 cube. Since the image can get rather large, rows are produced on the fly when a caller requests them. class ImageExpand (drpi.Image): def __init__ (self, source, stamps): sw = source.width sh = source.height ex = stamps [0].width ey = stamps [0].height drpi.Image.__init__ (self, sw * ex, sh * ey) self.source = source self.stamps = stamps self.source_width = sw self.source_height = sh self.expandx = ex self.expandy = ey self.row = bytearray (3 * self.width) print ("{} of {} bytes".format ( "ImageExpand", len (self.row) * self.height)) def withRow (self, r, action): rdiv, rmod = divmod (r, self.expandy) self.source.withRow (rdiv, lambda srcrow: self.withRow_one (srcrow, rmod)) action (self.row) def withRow_one (self, srcrow, rmod): stamps = self.stamps for k in range (0, len (srcrow), 3): kr = srcrow [k ] kg = srcrow [k + 1] kb = srcrow [k + 2] index = \ ((kr & 0x80) >> 5) | \ ((kg & 0x80) >> 6) | \ ((kb & 0x80) >> 7) stamp = stamps [index] stamp.withRow (rmod, lambda starow: self.withRow_piece (starow, k)) def withRow_piece (self, starow, k): pos = k * self.expandx self.row [pos : pos + len (starow)] = starow And here's a Lain dithered with serpentine Stucki dispersion and expanded with stamps of U+2665. Basically, DHALF.TXT allowed the production of a Lain dithered with love.

Been working on this for a couple weeks. https://github.com/seisatsu/Dennis Dennis is a MUD (Multi-User Dungeon, aka a multi-player text adventure) and collaborative writing exercise inspired by ifMUD, in which all content is created by the users, by utilizing in-game commands. The game starts with a single empty room, and one or more players build a world from that point by adding rooms, exits, and items, and describing them. The Github page has a link to a public test instance.

A month went by, so here's a C accelerator module for error diffusion dithering, using only the Python standard library. Everything but the pixel loop is kept in Python for convenience, only the pixel loop is moved to C for speed. First, filters >>10332 export their dispersion positions and tables in a way that is convenient for C code, in this case by using array.array objects. class ErrorFilter: def rows_below (self): pass def lateral (self): pass def disperse (self, column, error, error_rows, rows_delta, ltr): pass def accelerator (self): pass class ErrorTableFilter (ErrorFilter): def accelerator (self): have = getattr (self, "accelerator_data", None) if have is not None: return have data = {} tables = self.tables posltr = self.posltr posrtl = self.posrtl data ["posltrx"] = array.array ( "b", [x for x, y in posltr]) data ["posltry"] = array.array ( "b", [y for x, y in posltr]) data ["posrtlx"] = array.array ( "b", [x for x, y in posrtl]) data ["posrtly"] = array.array ( "b", [y for x, y in posrtl]) data ["tables"] = [ array.array ("h", t) for t in tables ] self.accelerator_data = data return data

The dithering engine >>10331 decides whether acceleration is available based on its constituent parts, mostly the nearest color search. The row processing function is then swapped out, so that there is no need to test for acceleration at the start of each row. class ErrorDither (EachRow): def __init__ (self, nearest, directions, filter): self.nearest = nearest self.directions = directions self.filter = filter self.filter_rows = 1 + filter.rows_below () self.lateral = filter.lateral () if accel.active and isinstance (nearest, ColorNearest8): self.accelerated = True self.accelerated_filter = filter.accelerator () self.eachrow_accel_func = accel.module.diffuse self.eachrow = self.eachrow_accel else: self.accelerated = False self.clamp_table = self.build_clamp_table (255, 255) def build_clamp_table (self, below, above): if self.accelerated: t = array.array ("B") else: t = [] t.extend ([0] * below) t.extend (range (256)) t.extend ([255] * above) return t def edit_before (self, image): self.build_error_rows (image.width) def build_error_rows (self, width): n = 3 * (width + 2 * self.lateral) if self.accelerated: init = [0] * n make = lambda: array.array ("h", init) else: make = lambda: [0] * n self.error_rows = [ make () for k in range (self.filter_rows) ] self.zero_row = make () def rotate_and_zero (self): errows = self.error_rows first = errows.pop (0) first [:] = self.zero_row errows.append (first) def eachrow_plain (self, row): ltr = next (self.directions) nearest = self.nearest.nearest disperse = self.filter.disperse clamp = self.clamp_table delta = 3 * self.lateral errows = self.error_rows errows0 = errows [0] if ltr: colrange = range (0, len (row), 3) else: colrange = range (len (row) - 3, -3, -3) for k in colrange: deltak = delta + k want = ( clamp [row [k ] + errows0 [deltak ] + 255], clamp [row [k + 1] + errows0 [deltak + 1] + 255], clamp [row [k + 2] + errows0 [deltak + 2] + 255], ) have = nearest (want) # row [k : k + 3] = have row [k ] = have [0] row [k + 1] = have [1] row [k + 2] = have [2] error = ( want [0] - have [0], want [1] - have [1], want [2] - have [2], ) disperse (k, error, errows, delta, ltr) self.rotate_and_zero () eachrow = eachrow_plain def eachrow_accel (self, row): ltr = next (self.directions) clamp = self.clamp_table delta = 3 * self.lateral errows = self.error_rows filter = self.accelerated_filter if ltr: posx = filter ["posltrx"] posy = filter ["posltry"] else: posx = filter ["posrtlx"] posy = filter ["posrtly"] self.eachrow_accel_func ( ltr, row, errows, delta, clamp, posx, posy, filter ["tables"]) self.rotate_and_zero ()

The C side has to arrange the Python arguments into a convenient form for processing. This is made more verbose by the need to clean up on every premature exit path. #define PAETHACCEL_DIFFUSE_MAX_ERROR_ROWS 8 #define PAETHACCEL_DIFFUSE_MAX_FILTER_TABLES 32 static void paethaccel_diffuse_release ( Py_buffer *bufs, Py_ssize_t count) { Py_ssize_t k; if (bufs == NULL) return; for (k = 0; k < count; k++) { PyBuffer_Release (&bufs [k]); } } static void paethaccel_diffuse_cleanup ( Py_buffer *row, Py_buffer *clamp, Py_buffer *posx, Py_buffer *posy, Py_buffer *bufs_error, Py_ssize_t len_error, Py_buffer *bufs_filter, Py_ssize_t len_filter) { PyBuffer_Release (row); PyBuffer_Release (clamp); PyBuffer_Release (posx); PyBuffer_Release (posy); paethaccel_diffuse_release (bufs_error, len_error); paethaccel_diffuse_release (bufs_filter, len_filter); } static int paethaccel_diffuse_getbuffers ( Py_buffer *bufs, int limit, PyObject *list, int flags, const char *label) { Py_ssize_t len, k; PyObject *item; len = PyList_GET_SIZE (list); if (len < 1) { PyErr_Format (PyExc_ValueError, "%s: no items", label); return 0; } if (len > limit) { PyErr_Format (PyExc_ValueError, "%s: too many items", label); return 0; } for (k = 0; k < len; k++) { /* borrowed reference */ item = PyList_GET_ITEM (list, k); if (PyObject_GetBuffer (item, &bufs [k], flags) < 0) { PyErr_Format (PyExc_ValueError, "%s: GetBuffer %zd", label, k); if (k > 0) { paethaccel_diffuse_release (bufs, k - 1); } return 0; } } return 1; } static int paethaccel_diffuse_lengths ( Py_ssize_t len_row, Py_ssize_t len_errow0, int delta, Py_ssize_t len_clamp, Py_ssize_t len_posx, Py_ssize_t len_posy, Py_ssize_t len_tables, Py_ssize_t len_table0) { Py_ssize_t target = (len_row + 2 * delta) * sizeof (signed short); if (len_errow0 != target) { PyErr_Format (PyExc_ValueError, "error row0 length: %zd expected: %zd", len_errow0, target); return 0; } target = 3 * 255 + 1; if (len_clamp != target) { PyErr_Format (PyExc_ValueError, "clamp length: %zd expected: %zd", len_clamp, target); return 0; } if (len_posx != len_posy) { PyErr_SetString (PyExc_ValueError, "pos x/y length mismatch"); return 0; } target = len_posx; if (len_tables != target) { PyErr_Format (PyExc_ValueError, "table count: %zd expected: %zd", len_tables, target); return 0; } target = (2 * 255 + 1) * sizeof (signed short); if (len_table0 != target) { PyErr_Format (PyExc_ValueError, "table0 length: %zd expected: %zd", len_table0, target); return 0; } return 1; } static PyObject * paethaccel_diffuse (PyObject *self, PyObject *args) { int ltr, delta; Py_buffer row, clamp, posx, posy; PyObject *error_rows, *filter_tables; Py_buffer buf_errows [PAETHACCEL_DIFFUSE_MAX_ERROR_ROWS]; Py_buffer buf_tables [PAETHACCEL_DIFFUSE_MAX_FILTER_TABLES]; Py_ssize_t k, len_errows, len_tables; signed short *data_errows [PAETHACCEL_DIFFUSE_MAX_ERROR_ROWS]; signed short *data_tables [PAETHACCEL_DIFFUSE_MAX_FILTER_TABLES]; /* PySys_WriteStdout ("inside paethaccel_diffuse\n"); */ if (! PyArg_ParseTuple ( args, "pw*O!iy*y*y*O!", &ltr, &row, &PyList_Type, &error_rows, &delta, &clamp, &posx, &posy, &PyList_Type, &filter_tables)) { return NULL; } len_errows = PyList_GET_SIZE (error_rows); len_tables = PyList_GET_SIZE (filter_tables); if (! paethaccel_diffuse_getbuffers ( buf_errows, PAETHACCEL_DIFFUSE_MAX_ERROR_ROWS, error_rows, PyBUF_WRITABLE, "error rows")) { paethaccel_diffuse_cleanup ( &row, &clamp, &posx, &posy, NULL, 0, NULL, 0); return NULL; } if (! paethaccel_diffuse_getbuffers ( buf_tables, PAETHACCEL_DIFFUSE_MAX_FILTER_TABLES, filter_tables, PyBUF_SIMPLE, "filter tables")) { paethaccel_diffuse_cleanup ( &row, &clamp, &posx, &posy, buf_errows, len_errows, NULL, 0); return NULL; } for (k = 0; k < len_errows; k++) { data_errows [k] = (signed short *) buf_errows [k].buf; } for (k = 0; k < len_tables; k++) { data_tables [k] = (signed short *) buf_tables [k].buf; } if (! paethaccel_diffuse_lengths ( row.len, buf_errows [0].len, delta, clamp.len, posx.len, posy.len, len_tables, buf_tables [0].len)) { paethaccel_diffuse_cleanup ( &row, &clamp, &posx, &posy, buf_errows, len_errows, buf_tables, len_tables); return NULL; } paethaccel_diffuse_work ( ltr, (unsigned char *) row.buf, row.len, data_errows, delta, (unsigned char *) clamp.buf, (signed char *) posx.buf, (signed char *) posy.buf, posx.len, data_tables); paethaccel_diffuse_cleanup ( &row, &clamp, &posx, &posy, buf_errows, len_errows, buf_tables, len_tables); Py_RETURN_NONE; } The explanation for the "pw*O!iy*y*y*O!" incantation is here: https://docs.python.org/3/c-api/arg.html

The pixel loop itself is a straightforward translation of the unaccelerated version, but with the nearest color search >>10181 and the filter's >>10332 dispersion inlined. static void paethaccel_diffuse_work ( int ltr, unsigned char *row, Py_ssize_t len_row, signed short **errows, int delta, unsigned char *clamp, signed char *posx, signed char *posy, Py_ssize_t len_pos, signed short **tables) { Py_ssize_t rowpos, rowpos_start, rowpos_inc; Py_ssize_t outer_k, outer_limit = len_row / 3; Py_ssize_t inner_k; signed short *errow0 = errows [0]; Py_ssize_t errowpos, errdxpos; signed char dx, dy; signed short *table; signed short *errowk; unsigned char want_r, want_g, want_b; int have_r, have_g, have_b; int erro_r, erro_g, erro_b; if (ltr) { rowpos_start = 0; rowpos_inc = 3; } else { rowpos_start = len_row - 3; rowpos_inc = -3; } rowpos = rowpos_start; for (outer_k = 0; outer_k < outer_limit; outer_k++) { errowpos = rowpos + delta; want_r = clamp [row [rowpos ] + errow0 [errowpos ] + 255]; want_g = clamp [row [rowpos + 1] + errow0 [errowpos + 1] + 255]; want_b = clamp [row [rowpos + 2] + errow0 [errowpos + 2] + 255]; have_r = (0x100 - ((want_r & 0x80) >> 7)) & 0xFF; have_g = (0x100 - ((want_g & 0x80) >> 7)) & 0xFF; have_b = (0x100 - ((want_b & 0x80) >> 7)) & 0xFF; row [rowpos ] = (unsigned char) have_r; row [rowpos + 1] = (unsigned char) have_g; row [rowpos + 2] = (unsigned char) have_b; erro_r = want_r - have_r + 255; erro_g = want_g - have_g + 255; erro_b = want_b - have_b + 255; for (inner_k = 0; inner_k < len_pos; inner_k++) { dx = posx [inner_k]; dy = posy [inner_k]; table = tables [inner_k]; errdxpos = errowpos + 3 * dx; errowk = errows [dy]; errowk [errdxpos ] += table [erro_r]; errowk [errdxpos + 1] += table [erro_g]; errowk [errdxpos + 2] += table [erro_b]; } rowpos += rowpos_inc; } }

Standard boilerplate for the C module, header: #define PY_SSIZE_T_CLEAN #include <Python.h> and footer: static PyMethodDef PaethAccelMethods [] = { {"diffuse", paethaccel_diffuse, METH_VARARGS, "error diffusion for one row"}, {NULL, NULL, 0, NULL} /* Sentinel */ }; static struct PyModuleDef paethaccelmodule = { PyModuleDef_HEAD_INIT, "paethaccel", /* name of module */ NULL, /* module documentation, may be NULL */ -1, /* size of per-interpreter state of the module, or -1 if the module keeps state in global variables. */ PaethAccelMethods }; PyMODINIT_FUNC PyInit_paethaccel (void) { return PyModule_Create (&paethaccelmodule); } The setup.py needed for the build command: import distutils.core as duco module = duco.Extension ( name = 'paethaccel', sources = ['paethaccel.c']) duco.setup ( name = 'paethaccel', version = '1.0', description = 'Paeth accelerator', ext_modules = [module]) A thin import module to make access to the accelerator uniform from Python code: try: import accel.paethaccel as module active = True print ("[paethaccel active]") except ImportError: active = False

The point of all this is in the timings. First an unaccelerated run in vanilla Python on the usual Lain >>9374, with the profiling overhead included, using serpentine Floyd-Steinberg: 530408 function calls (529534 primitive calls) in 2.161 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 249000 1.143 0.000 1.143 0.000 dither.py:1150(disperse) 600 0.658 0.001 2.032 0.003 dither.py:989(eachrow_plain) 249000 0.217 0.000 0.217 0.000 dither.py:896(nearest) 1200 0.080 0.000 0.080 0.000 {method 'compress' of 'zlib.Compress' objects} 600 0.013 0.000 0.013 0.000 dither.py:982(rotate_and_zero) 81 0.006 0.000 0.006 0.000 {method 'decompress' of 'zlib.Decompress' objects} The pixel loop is in eachrow_plain, which takes 2.032 seconds over the 600 rows of the image. The accelerated version: 32378 function calls (31502 primitive calls) in 0.142 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1200 0.080 0.000 0.080 0.000 {method 'compress' of 'zlib.Compress' objects} 600 0.012 0.000 0.012 0.000 {built-in method diffuse} 81 0.006 0.000 0.006 0.000 {method 'decompress' of 'zlib.Decompress' objects} The pixel loop is now in diffuse, and takes 0.012 seconds for the entire image. The dithering pass is now faster than the Python built-in zlib compression that produces the IDAT stream for png output. The speedup factor decreases as the complexity of the filter increases. The same image with unaccelerated serpentine Stucki: 530470 function calls (529596 primitive calls) in 3.811 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 249000 2.779 0.000 2.779 0.000 dither.py:1150(disperse) 600 0.667 0.001 3.680 0.006 dither.py:989(eachrow_plain) 249000 0.221 0.000 0.221 0.000 dither.py:896(nearest) 1200 0.080 0.000 0.080 0.000 {method 'compress' of 'zlib.Compress' objects} 600 0.012 0.000 0.013 0.000 dither.py:982(rotate_and_zero) 81 0.006 0.000 0.006 0.000 {method 'decompress' of 'zlib.Decompress' objects} The pixel loop takes 3.680 seconds. With acceleretion: 32440 function calls (31564 primitive calls) in 0.160 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 1200 0.079 0.000 0.079 0.000 {method 'compress' of 'zlib.Compress' objects} 600 0.030 0.000 0.030 0.000 {built-in method diffuse} 81 0.006 0.000 0.006 0.000 {method 'decompress' of 'zlib.Decompress' objects} The dithering time is 0.030 seconds, still faster than the zlib compression. The price is that the C version is not as flexible nor as friendly to debugging as the Python version.

>>5677 I wish I had a choice. I wish I had a thread to vent all my python hate

What are some good books dealing with advanced python? Specially looking for something that covers decorators in detail. I have read some of Fluent Python, and I find it quite good, but everything I have read so far on decorators, would not give me any tips on how I could use them.

>>11335 >but everything I have read so far on decorators, would not give me any tips on how I could use them Here's a pass-through decorator that registers a parser in a table by name: class ParserRegistry: def _adder (store): def named (name): def deco (cls): store [name] = cls return cls return deco return named STORE = {} ADD = _adder (STORE) @ staticmethod def parser (name): return ParserRegistry.STORE [name] () @ ParserRegistry.ADD ('drand') class DitherRandomParser (Parser): def editor (self, params): return DitherRandom () Here's another decorator that compiles struct formats into structs and stores them as class fields: def _structs_from_formats (cls): prefix = "STRUCT_FORMAT_" prelen = len (prefix) items = [ (k [prelen:], v) for k, v in vars (cls).items () if k.startswith (prefix) ] add = "STRUCT_" for name, fmt in items: setattr (cls, add + name, struct.Struct (fmt)) return cls @ _structs_from_formats class C: STRUCT_FORMAT_LSD = "< 2H 3B" STRUCT_FORMAT_IMGDESC = "< 4H 1B" STRUCT_FORMAT_GRAPHTROL = "< 1B 1H 1B" And a decorator that saves a context before some action and restores it after: class SaveRestore: @ staticmethod def exec (ctx, saverestore, callthis, callargs): if saverestore: ctx.save () callthis (* callargs) ctx.restore () else: matrix = ctx.get_matrix () callthis (* callargs) ctx.set_matrix (matrix) @ staticmethod def wrap (func): def f (* args): ctx = args [1] sr = args [0].saverestore SaveRestore.exec (ctx, sr, func, args) return f class RectTransformContext (Rect): def __init__ (self, target, saverestore): self.target = target self.saverestore = saverestore def transform (self, ctx, x, y, dx, dy): pass @ SaveRestore.wrap def work (self, ctx, x, y, dx, dy): self.transform (ctx, x, y, dx, dy) self.target.rect (ctx, x, y, dx, dy) def rect (self, ctx, x, y, dx, dy): self.work (ctx, x, y, dx, dy) class RectFixedTilt (RectTransformContext): def __init__ (self, target, saverestore, angle): RectTransformContext.__init__ (self, target, saverestore) self.angle = angle def transform (self, ctx, x, y, dx, dy): kge.tilt_rectangle (ctx, self.angle, x, y, dx, dy)

>>11335 i don't understand why you need a book. they're not that complicated. decorators allow you to change def functionModifier(function): bla bla return blablabla def function_1(args): ble ble return blebleble function_1 = functionModifier(function_1) to this: def functionModifier(function): bla bla return blablabla @functionModifier def function_1(args): ble ble return blebleble It allows you to remove code duplication when it comes up like this. Use decorators where you would use the subsequent pattern in a more functional language. def functionModifier(function): bla bla return blablabla function_1 = functionModifier( Lambda args: ble ble return blebleble ) Since Python lamdae can only contain one statement, this isn't always possible in python. If you understand this, but still get confused by the way some people use decorators (which is very understandable), i would suggest you focus on the patterns they're using and why rather than the syntax they are using. I guess you could get that from a book, but i feel like a few examples would solve it.

I'm creating functions for parsing text and generating tables and documentation. My hope is to read in different project files and create a README.txt file for git repositories. def findBrief(comment): pat = re.compile("(@[bB]rief)([a-zA-Z .\n#]*)") match = pat.search(comment) if match: match = match.group(2) match = match.strip().split('\n') for n , line in enumerate(match): if line[0] is "#": match[n] = match[n][1:] match = "".join(match) return match else: return None def findInfo(comment): pat = re.compile("(@[iI]nput)([a-zA-Z .\n#:-]*)") match = pat.search(comment) if match: match = match.group(2) match = match.strip().split('\n') for n , line in enumerate(match): if line[0] is "#": match[n] = match[n][1:] match = "".join(match).strip().split("-") match = filter(None,match) for n, tmp_match in enumerate(match): match[n] = match[n].strip() return match else: return None def findReturn(comment): pat = re.compile("(@[rR]eturn)([a-zA-Z .\n#]*)") match = pat.search(comment) if match: match = match.group(2) match = match.strip().split('\n') for n , line in enumerate(match): if line[0] is "#": match[n] = match[n][1:] match = "".join(match).replace("#","") return match else: return None def findLicense(comment): pat = re.compile("(@[lL]icense)([a-zA-Z .\n#]*)") match = pat.search(comment) if match: return True else: return False class DocSubject: docInput = "" docBrief = "" docBeturn = "" docLicense = "" def setBrief(self,inputBrief=""): self.docBrief = inputBrief def printBrief(self): print self.docBrief def setInput(self,inputInfo=''): self.docInput = inputInfo def printInput(self): print self.docInput def setReturn(self,returnInfo=''): self.docReturn = returnInfo def printReturn(self): print self.docReturn def setLicense(self,inputLicense): self.docLicense = inputLicense def printLicesnse(self): print self.docLicense def createTable(docSubjectList): table = "<table style='width:100%'><tr><th>Name</th><th>Brief</th><th>Inputs</th><th>Returns</th></tr>" for docSubject in docSubjectList: if docSubject.docLicense is False: table = table + "<tr><td>placeHolder</td><td>" + str(docSubject.docBrief) + "</td><td>" for inputs in docSubject.docInput: table = table + str(inputs) + "<br/>" table = table + "</td><td>" + str(docSubject.docReturn) + "</td></tr>" table = table + "</table>" print table f = open(userFile, "r").readlines() comments = getCommentBlocks(f) docSubjectList = [] n = 0 for n , comment in enumerate(comments): docSubjectList.append(DocSubject()) docSubjectList[n].setLicense(findLicense(comment)) if docSubjectList[n].docLicense is False: docSubjectList[n].setBrief(findBrief(comment)) docSubjectList[n].setInput(findInfo(comment)) docSubjectList[n].setReturn(findReturn(comment)) createTable(docSubjectList)

>>11596 Recompiling the same regexp on every function call defeats the purpose of compiling.

>>11599 But its not the same regex. Each time im looking for a new work in the same string [@input, @return, @brief,@license] Im not sure if there is a way to switch this up beteeen the different search methods.

is it possible to make a simple Quake-like game in Python that doesn't consume too many resources? I know a lot of OpenGL, C, and Python, but have never used them together. I was thinking of using Cython to make that game.

>>11601 I think "possible" is too loose a term actually. What I'm asking is whether such a game could be efficient and workable.

>>11600 >But its not the same regex. In your main loop >>11596 >for n , comment in enumerate(comments): you repeatedly call >findLicense(comment) >findBrief(comment) >findInfo(comment) >findReturn(comment) Each of these four functions uses its own constant regexp, and compiles it on each call.

>>11596 A main method and a "__main__" test would be a good idea. def main (): ... if __name__ == "__main__": main () https://docs.python.org/3/tutorial/modules.html#executing-modules-as-scripts >f = open(userFile, "r").readlines() Always wrap 'open' in a 'with' block. with open (userFile, "r") as userf: f = userf.readlines () https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files A comment to DocSubject conversion function and a list comprehension would clean up your main loop. def build_DocSubject_from_comment (comment): ... def main (): ... docSubjectList = [ build_DocSubject_from_comment (comment) for comment in comments ] ... https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

>>11604 That makes alot of sense thanks for pointing that out Present Day, Present Time! AHAHAHAHAHA!

>>11596 >class DocSubject: >docInput = "" >docBrief = "" >docBeturn = "" >docLicense = "" These create class attributes rather than instance attributes -- make sure this is what you want, especially given docBeturn [sic] versus self.docReturn. >print self.docBrief >print self.docInput >print self.docReturn >print self.docLicense >print table This is explicitly a "Python 3 Thread" >>13. https://docs.python.org/3/whatsnew/3.0.html#print-is-a-function

>>11607 some of the servers I work with are old centOS 5 servers so I tend to stay away from python 3. Although this project has no need to ever run on one of our servers. I should make it run on python 3 it would not be hard. [/code] briefSearch = re.compile("(@[bB]rief)([a-zA-Z .\n#]*)") inputSearch = re.compile("(@[iI]nput)([a-zA-Z .\n#:-]*)") licenseSearch = re.compile("(@[lL]icense)([a-zA-Z .\n#]*)") returnSearch = re.compile("(@[rR]eturn)([a-zA-Z .\n#]*)") for n , comment in enumerate(comments): docSubjectList.append(DocSubject()) docSubjectList[n].setLicense(findLicense(comment, licenseSearch)) if docSubjectList[n].docLicense is False: docSubjectList[n].setBrief(findBrief(comment,briefSearch)) docSubjectList[n].setInput(findInfo(comment, inputSearch)) docSubjectList[n].setReturn(findReturn(comment, returnSearch)) createTable(docSubjectList) [/code]

>>11596 If you don't want to use a templating engine like jinja2 to build the table, at least use python's built in string formatting features. >>11601 >>11602 EVE Online uses a custom version of python for most of the game logic. I don't think it includes anything that needs to happen in real-time.

>>11612 Is this better Present Day, Present Time! AHAHAHAHAHA! sometimes I forget that features exist. def createTable(docSubjectList): table = "<table style='width:100%'><tr><th>Name</th><th>Brief</th><th>Inputs</th><th>Returns</th></tr>" for docSubject in docSubjectList: if docSubject.docLicense is False: table = Template("$preTable<tr><td>placeHolder</td><td>$docBrief</td><td>").substitute(preTable=table, docBrief=docSubject.docBrief) for inputs in docSubject.docInput: table = Template("$preTable $docInput<br/>").substitute(preTable=table,docInput=inputs) table = Template("$preTable</td><td>$docReturn</td></tr>").substitute(preTable=table,docReturn=docSubject.docReturn) table = Template("$preTable</table>").substitute(preTable=table) print table

>>11605 I like a lot of your ideas. I feel like my code is much cleaner now. (edited: I somehow pasted the wrong code.) def build_DocSubject_from_comment(comment, briefSearch, inputSearch, licenseSearch, returnSearch): obj = DocSubject() obj.setLicense(findLicense(comment,licenseSearch)) if obj.docLicense is False: obj.setBrief(findBrief(comment,briefSearch)) obj.setInput(findInfo(comment,inputSearch)) obj.setReturn(findReturn(comment,returnSearch)) return obj def main(): userFile = checkForArgs() if validateFile(userFile): print "start" else: exit() with open(userFile, "r") as f: readFile = f.readlines() comments = getCommentBlocks(readFile) docSubjectList = [] briefSearch = re.compile("(@[bB]rief)([a-zA-Z .\n#\(\)'\",]*)") inputSearch = re.compile("(@[iI]nput)([a-zA-Z .\n#:-]*)") licenseSearch = re.compile("(@[lL]icense)([a-zA-Z .\n#]*)") returnSearch = re.compile("(@[rR]eturn)([a-zA-Z .\n#:-]*)") docSubjectList = [ build_DocSubject_from_comment(comment, briefSearch, inputSearch, licenseSearch, returnSearch) for comment in comments ] createTable(docSubjectList) if __name__ == "__main__": main()

>>11596 >if line[0] is "#": The fact that this works and the character range over which it works are both implementation details. Use == instead: >>> 'λx.xx' [0] is 'λ' False >>> 'λx.xx' [0] == 'λ' True https://docs.python.org/3/reference/expressions.html#value-comparisons https://docs.python.org/3/reference/expressions.html#is-not >for n , line in enumerate(match): >if line[0] is "#": >match[n] = match[n][1:] While this particular use case will work, performing any mutating operation on a collection while iterating over it is a bad habit to get into. This can be replaced with: match = map (lambda line: line [1:] if line [0] == '#' else line, match) and the subsequent join will work unchanged. https://docs.python.org/3/library/functions.html#map https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions >match = filter(None,match) >for n, tmp_match in enumerate(match): >match[n] = match[n].strip() This is: match = [ item.strip () for item in match if item ] https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions >if match: >return True >else: >return False This is just: return bool (match) https://docs.python.org/3/library/functions.html#bool >>11616 >docSubjectList = [] This is not needed.

>>11618 >>11618 Thanks a lot! I have learnt so much in the past few days. I think this is the fastest I have picked stuff up in a long time. I remenber having issuse with value comparisons in the past and never knew why it was happening. I feel like I have a much better grasp on the subject. I also struggled with lambda but put in context of something I am working on it now seems simple. def findBrief(comment, pat): match = pat.search(comment) if match: match = match.group(2) match = match.strip().split('\n') match = map(lambda line: line[1:] if line[0] == "#" else line, match) match = "".join(match) return match else: return None def findInput(comment, pat): match = pat.search(comment) if match: match = match.group(2) match = match.strip().split('\n') match = map(lambda line: line[1:] if line[0] == "#" else line, match) match = "".join(match).strip().split("-") match = filter(None,match) match = [ item.strip() for item in match if item] return match else: return None def findReturn(comment, pat): match = pat.search(comment) if match: match = match.group(2) match = match.strip().split('\n') match = map(lambda line: line[1:] if line[0] == "#" else line, match) match = "".join(match).replace("#","") return match else: return None The next thing I want to work on is a file loader thing. The idea is all in my head still. The basic idea is to build a directory tree out of objects. Then feed a directory into a queue for processing. Kinda like in my pic.

>>11627 You are making nice progress. >match = filter(None,match) This is not needed. The "if item" part of the list comprehension performs this filtering. >>11596 >>11614 This table creation doesn't seem complex enough to warrant complicating it with templates, and certainly not with recreating constant templates in a loop. Regardless, what it does need is a separate function that converts one docSubject into a table row. The createTable function can perform the overall assembly of the table header, body and footer, without having to know the details of row creation. def build_tablerow_from_DocSubject (docSubject): ... def createTable (docSubjectList): ... body = "".join ( build_tablerow_from_DocSubject (docSubject) for docSubject in docSubjectList if docSubject.docLicense is False ) ... # assemble table header, body and footer ... A generator expression will be more memory efficient here than a list comprehension because the join does not need all rows at the same time, it only needs them one by one. https://docs.python.org/3/tutorial/classes.html#generator-expressions

>>11627 >build a directory tree out of objects. Then feed a directory into a queue for processing See if os.walk might be a building block you could use. https://docs.python.org/3/library/os.html#os.walk And the glob module: https://docs.python.org/3/library/glob.html#glob.iglob

>>11628 def createTable(docSubjectList): body = "<table style='width:100%'><tr><th>Name</th><th>Brief</th><th>Inputs</th><th>Returns</th></tr>" body = body + "".join( buildTableRowFromDocSubject(docSubject) for docSubject in docSubjectList if docSubject.docLicense is False) body = body + "</table>" print body def buildTableRowFromDocSubject(docSubject): row = "<tr><td>placeHolder</td><td>%(docBrief)s</td><td>" % {"docBrief": docSubject.docBrief} for inputs in docSubject.docInput: row = "%(row)s%(docInput)s<br/>" %{"row":row, "docInput":inputs} row = "%(row)s</td><td>%(docReturn)s<td/><tr/>" % {"row":row ,"docReturn": docSubject.docReturn} return row Like this. It is much cleaner and easier to read so I that is a plus.

>>11633 That is the cleaner way to structure it, so good job, Lain.

>>11344 >>11349 I just wrote my first decorator. Your posts were really helpful. Thanks a lot.

>>11633 Required notice. String interpolation for HTML is error prone at best and a security hazard at worst. Imagine how much would break if 'docBrief' included an angle bracket.

>>11759 Take a closer look at the character classes in the four regexps.

I have been swamped at work and this project has had to go on the back burner for a while. >>11764 Lain (>>11759) does make a good point. The fact that my regex is not matching angel brackets right now is an oversight. In the future, I would like to include almost all special characters. My guess would be that I need to HTML encode these values during table creation. Something else to get around to. Thanks for looking out. :)

>>13 What book should I pick up/Where should I go once I've gone through the equivalent of an 'intro' course? I guess 'intermediate' is the right word, but I'm just a bit less clueless than when I started.

>>12360 Don't pick up any books, do a project, you hopefully started learning the basics of python with the intention to do something with that knowledge right? Start doing that thing, if you face a roadblock, then do more research, but not until then. Theory then practice, not theory then more theory.

>>12360 http://www.composingprograms.com/

>>12360 I agree with >>12361 here. You will hit the theory roadblock sooner or later anyway, when you start getting into more advanced topics. Better focus on actually learning to program first, so you'll have basics for theory later

>>12361 >>12363 >>12362 So basically use documentation to fill the gaps in a project I'm working on and learn from there. If I'm totally stuck, find a relevant book and work through it and apply what I learned to that project? Honestly, that's excellent advice (and I feel a bit dull not thinking of doing that earlier). Thank you!

I am using Emacs as a Python IDE. I see that the refactoring library is rope, but the documentation on its github[0] page is lacking. does anyone have a good tutorial on rope? ideally not video. Thanks [0] https://github.com/python-rope/rope

What's the best way to make python syntax tolerable? Preferably I'd like something like def in the c language where you can easily substitute functionality

I am currently working on an application to manage my anime images. This is also my first GUI project. Suggestions would be appreciated https://github.com/mananapr/akari

>>12822 But why? You can write in C if you want to. https://docs.python.org/3.7/extending/extending.html

Did anyone see the changes they are making for python 3.8 Python is becoming less beginner friendly. :=, @, what's next $

>>14367 @ is just a new (3.5) operator. It actually simplifies things because it clarifies the use of *. := is a different beast. It adds a new expression to the grammar, it's not immediately clear why it's needed, and it doesn't support the same syntax as regular assignment statements. And that's not even getting into the PEP that had to be rammed through to make it happen. I'm hoping that Python 4 pares down the core language features.

>>36 in python 3.7 there is type-hinting which is insanely cool. So for your new projects you can hint whenever you want. Generally just writing good comments for functions and the syntax is the only way to know for older projects, unfortunately.

Apparently, random number generators don't work in my device sometimes. Whenever I try to use the python 3's random.randint function it loads infinitely, and the same thing happened when I tried to create my own random.randint based on the time and path to the program. What do i do? I remember that the same thing happened when i tried to use the lua's math.random. code: python class com: def pick(self, game): while True: if not game.pick(self, randint(0, 2), randint(0, 2)): continue break class game: def __init__(self, p1, p2): self.players = { p1: 'X', p2: 'O' } self.board = [ ['1', '2', '3'], ['4', '5', '6'], ['7', '8', '9'] ] #... def pick(self, player, line, column) -> bool: if self.board[line][column] not in [str(n) for n in range(1, 11)] or player not in self.players.keys(): return False self.board[line][column] = self.players[player] return True $ python Python 3.7.4 (default, Jul 11 2019, 08:22:34) [Clang 8.0.7 (https://android.googlesource.com/toolchain/clang b55f2d4ebfd35bf6 on linux Type "help", "copyright", "credits" or "license" for more information. >>> import classes >>> p1 = classes.com() >>> p2 = classes.com() >>> j = classes.game(p1, p2) >>> p1.pick(j) and this happens when i use ctrl+c: [code] ^CTraceback (most recent call last): File "<stdin>", line 1, in <module> File "/data/data/com.termux/files/home/programas/python/pyhash/classes.py", line 10, in pick if not game.pick(self, randint(0, 2), randint(0, 2)): File "/data/data/com.termux/files/usr/lib/python3.7/random.py", line 222, in randint return self.randrange(a, b+1) File "/data/data/com.termux/files/usr/lib/python3.7/random.py", line 174, in randrange def randrange(self, start, stop=None, step=1, _int=int): KeyboardInterrupt [code]

>>15078 If random.randint hangs it should exhibit that behavior when called directly in a fresh REPL without 'import classes'.

I have to say, I'm a python newbie, and I've only been writing python for a couple of years now. At first I found it insufferable (As someone with a foundation in C) but what I found really nice, was being able to easily create dictionaries, and use them as argument passing, allowing to have go-like multiple return functions. e.g. def foo(args): if args.get("bar") is None: return { "status": -1, "reason": "invalid argument" } print(args.get("bar")) return { "status": 0 } def main(): payload = { "bar": "This is a test" } res = foo(payload) print(res) if __name__ == "__main__": main() Is this good design? I have been using pyls with vim (although I don't have it on this computer). I am trying to make code "pythonic" but I really am making things up as I go along, and trying to follow the language server warnings.

>>15199 Couple of WEEKS. Sorry.

>>15199 Using namedtuples, plain tuples, dictionaries or the like for multiple return values is perfectly fine. Just try to avoid it in a tight inner loop because of the repeated allocations. As for receiving the arguments in a function, python has a keyword arguments mechanism that you should be aware of, so that you can make your own decision about whether it's more appropriate for your particular use case. def foo (**kwargs): ... def main (): payload = { "bar": "This is a test" } res = foo (**payload) # alternatively, without payload res = foo (bar = "This is a test") https://docs.python.org/3/reference/compound_stmts.html#function-definitions

>>15199 > if args.get("bar") is None Why not "if not args.get("bar")"? None's bool is false.

>>15203 I did not know this thank you =).

>>15203 ... if not args.get ("bar"): destroy_the_world () ... foo ({"bar": 0})

>>15199 >>15203 >>15207 The idiomatic way would be: if 'bar' not in args:

>>15215 Oh cool this is even better thanks.

>>15215 You then perform a second lookup to get the value. If you can guarantee a signaling value that is not in the dictionary, the second argument to 'get' allows you to avoid duplicating the lookup work.

I can't find a mobile tetris clone that has my requirements, so i will try to create my own tetris with pygame_sdl. I just downloaded a PDF about it, but i can't say that it's good. Is there any good PDFs about pygame?

>Python 3 Thread >share projects you have been working on One way to generate a Hilbert space-filling curve is to recursively expand a start symbol using the production rules of a context-free grammar: class TestHilbert (PathTest): def defaultConfig (self): # dir: up, down, left, right # side: left, right, top, bottom expand = { "UL": ("RB", "UL", "UL", "LT"), "RB": ("UL", "RB", "RB", "DR"), "LT": ("DR", "LT", "LT", "UL"), "DR": ("LT", "DR", "DR", "RB"), "RT": ("DL", "RT", "RT", "UR"), "DL": ("RT", "DL", "DL", "LB"), "UR": ("LB", "UR", "UR", "RT"), "LB": ("UR", "LB", "LB", "DL") } # (dx, dy) wind = { "UL": ((0, 1), (0, 0), (1, 0), (1, 1)), "RB": ((0, 1), (1, 1), (1, 0), (0, 0)), "LT": ((1, 0), (0, 0), (0, 1), (1, 1)), "DR": ((1, 0), (1, 1), (0, 1), (0, 0)), "RT": ((0, 0), (1, 0), (1, 1), (0, 1)), "DL": ((0, 0), (0, 1), (1, 1), (1, 0)), "UR": ((1, 1), (1, 0), (0, 0), (0, 1)), "LB": ((1, 1), (0, 1), (0, 0), (1, 0)) } size = 512 return { "expand" : expand, "wind" : wind, "start" : "RT", "level" : 5, "closepath": False, "width" : size, "height" : size, "size" : size } def makePoints (self, cfg): return TestHilbert.points ( cfg, 0, 0, cfg ["size"], cfg ["start"], cfg ["level"]) @ staticmethod def points (cfg, x, y, size, var, level): s12 = size // 2 delta = cfg ["wind"] [var] if level == 0: s14 = size // 4 return [ (x + dx * s12 + s14, y + dy * s12 + s14) for dx, dy in delta ] symbols = cfg ["expand"] [var] level -= 1 return itertools.chain.from_iterable ([ TestHilbert.points ( cfg, x + d [0] * s12, y + d [1] * s12, s12, sym, level ) for d, sym in zip (delta, symbols) ]) This can be combined with the shape painter mask >>14824 and the pairing function >>13784 to linearly interpolate the position of a mask circle along each segment: class ShapeFillData: @ staticmethod def hilbert (data): w = data ["width" ] h = data ["height"] time = data ["time" ] size = data ["size" ] level = data ["level" ] radius = 11 steps = 12 hil = dtes.TestHilbert () cfg = hil.defaultConfig () cfg ["width" ] = size cfg ["height"] = size cfg ["size" ] = size cfg ["level" ] = level pos = hil.makePoints (cfg) starmap = itertools.starmap pairs = ither.pairs inter = ShapeFillTools.interpolate def f1 (a, b): return inter (a, b, time, steps) return list ( starmap (lambda p1, p2: ("circle", (f1 (p1 [0], p2 [0]), f1 (p1 [1], p2 [1]), radius), 1), pairs ( starmap (lambda x, y: (x * w / size, y * h / size), pos)))) class MakeMask_sfhilbert (MakeMaskShapeFill): def __init__ (self): MakeMaskShapeFill.__init__ (self, { "size" : 512, "level": 3}, ShapeFillData.hilbert) As an example here is a stream of Lain bubbles flowing along the Hilbert curve.

>>36 >>36 You have 'type' and 'isinstance' as your best friends if you're in doubt. For defensive programming, you can always wrap with 'try...except...' or 'if' statement.

from humble bundle

python user_choice = input('''press... 1 to try again... 2 to download the next problem... anything else to quit the program... --> ''').strip() if user_choice == '1': continue elif user_choice == '2': break else: cancel = True break How do I turn this code into a function and break a loop through it? I don't want to turn the input into a function and analyze the return of it every time I use the function.

>>15493 Why not? That would be the natural thing to do... But I suppose you *could* have a function that throws an exception and have the try/catch statement that uses it break. Or I suppose you could implement a State Machine via returning the next function to run but... Just use return values.

>>15494 The only thing that will change is that the input will not repeat itself, but I will have to evaluate the input every time I call the function. >But I suppose you *could* have a function that throws an exception and have the try/catch statement that uses it break. I had thought of doing it, but I have three conditions. Maybe if I did an exception class with a property that identifies the class.

>>15498 Just tried the exception thing: python class DontTryAgain(Exception): def __init__(self, user_opt): self.option = user_opt def getchoice(): user_choice = input('''press... 1 to try again... 2 to download the next problem... anything else to quit the program... --> ''').strip() if user_choice != '1': raise DontTryAgain(user_choice) # ... try: getchoice() continue except DontTryAgain as e: cancel = False if e.option == '2' else True break site = bs(site.content, 'html.parser') except Exception as e: print(f'\nexception: {e}') try: getchoice() continue except DontTryAgain as e: cancel = False if e.option == '2' else True break I think it got cleaner, but it's still very repetitive.

>>15493 The 'obvious' choice would be to make a function that will retry the downloads for you. #!/bin/env python def run_with_retries( function ): try: return function except: user_choice = ... if user_choice == '1': return run_with_retries( function ) elif user_choice == '2': return else: raise UserCancelldException() def download_problem_1(): ... problem1 = run_with_retries( download_problem_1 ) The annoyance with this is that you have to give each function a name even if it's just a couple lines because Python's lambdas are not very strong.c

>>15511 s/return function/\0()/

>>15511 I don't get it well. It tries to return the function in the parameter; if an exception appears the input is called; if the input is '1' the function tries to return again; if the input is '2' the function returns None; else the function raises the exception? >The annoyance with this is that you have to give each function a name even if it's just a couple lines I think I can avoid that by creating a class representing a challange, but it would be simpler to repeat the code.

Hello, I am trying to use the Python Mock library to mock two subprocess calls a function makes. I would like this call to return different data depending on the time of API call I make in my subprocess. I posted below the subprocess calls that I am making within my function. whmapicall = subprocess.Popen(["whmapi1" , "showbw", 'searchtype=user', 'search=^%s$' % self.username], stdout=subprocess.PIPE) whmapireturn = whmapicall.stdout.read().split("\n") whmapicall = subprocess.Popen(["whmapi1" , "getpkginfo", "pkg=%s" % self.plan], stdout=subprocess.PIPE) whmapireturn = whmapicall.stdout.read().split("\n") [code] The issue is I am not sure how I would handle the inheritance. Here is my attempt at mocking the subprocess library. [code] class MockedPopen: callargs = None PIPE = None def __init__(self, args, **kwargs): self.args = args self.returncode = 0 def __enter__(self): return self def __exit__(self, exc_type, value, traceback): pass @classmethod def Popen(self, cmd , **kwargs): return self if args[1] == 'getpkginfo': class stdout(): def read(self): return '''--- data: pkg: BWLIMIT: 14 CGI: 1 CPMOD: paper_lantern DIGESTAUTH: 'n' FEATURELIST: default FRONTPAGE: 0 HASSHELL: 0 IP: 0 LANG: en MAXADDON: 0 MAXFTP: ~ MAXLST: ~ MAXPARK: 0 MAXPOP: ~ MAXSQL: ~ MAXSUB: ~ MAX_DEFER_FAIL_PERCENTAGE: unlimited MAX_EMAILACCT_QUOTA: unlimited MAX_EMAIL_PER_HOUR: unlimited QUOTA: 100 _PACKAGE_EXTENSIONS: '' metadata: command: getpkginfo reason: OK result: 1 version: 1 ''' if args[1] == "showbw": class stdout(): def read(self): return '''--- data: acct: - bwusage: - deleted: 0 domain: test.edwinmundo.dev usage: 0 deleted: 0 limit: 104857600 maindomain: test.edwinmundo.dev owner: root reseller: 0 totalbytes: 0 user: edwin month: 10 reseller: root totalused: 47803 year: 2019 metadata: command: showbw reason: OK result: 1 version: 1''' @mock.patch('cpquotafix.subprocess', MockedPopen) def testUsername(self): user = cpquotafix.User("eddy","cthulhu","idecfunhouse.com","root", None ,"10000",None,None,None,None,None,None,None) assert user.username == "eddy"

>>15513 > It tries to return the function in the parameter see >>15512 It tries to run the function that will download the current problem. I should've named the variable 'function' as 'download_problem'. So, that line means "run the download_problem function and return the result" >if an exception appears the input is called Yes, if an exception is raised, that means the download function failed, so it asks the user what to do. >if the input is '1' the function tries to return again; Not exactly. What that line means is that the /entire/ process is restarted. So it basically just goes back to the beginning of the function there. This means that if the download function fails again, it will again ask the user what to do. This technique of calling a function from within itself is called recursion. You could do the same thing with a loop here, but i think calling the function again shows better that it's just going to try downloading again. >if the input is '2' the function returns None; '2' means that the user wants to cancel this download. I decided to have it return None to show that the user cancelled. You might want it to return something else, but idk how exactly you are going to use this. >else the function raises the exception? Yes. If the user input something other than '1' or '2', then the function throws an exception, which will cause the program to stop. You can catch that exception in a big block that includes all of your code if you want it to do something else when it exits. >I think I can avoid that by creating a class representing a challange, but it would be simpler to repeat the code. Looking back, i think maybe this point might not be relevant for your use case, and i might not've explained it well. If you use the same function to download each problem, you're fine.

>>15515 >None ,"10000",None,None,None,None,None,None,None) Use default argument values because you are starting to look like winapi. https://docs.python.org/3/tutorial/controlflow.html#default-argument-values >class MockedPopen: You do not need manual stub objects. You can mock the read() directly with side_effect. https://docs.python.org/3/library/unittest.mock-examples.html#side-effect-functions-and-iterables import subprocess import unittest.mock as mock def apicalls (): pop = subprocess.Popen (['a', 'b', 'c']) ret = pop.stdout.read ().split ("\n") print (ret) pop = subprocess.Popen (['d', 'e', 'f']) ret = pop.stdout.read ().split ("\n") print (ret) def popen_args_from_apicalls (m): # print (m.mock_calls) call = m.mock_calls [-2] name, args, kwargs = call return args def read_with_args (args): lst = args [0] return "return value for\nread of Popen\nwith {}".format (lst) def fakeread (m): args = popen_args_from_apicalls (m) return read_with_args (args) @ mock.patch ('__main__.subprocess.Popen') def test1 (m): m.return_value.stdout.read.side_effect = lambda *args: fakeread (m) apicalls () def main (): test1 () if __name__ == "__main__": main () Customize read_with_args to your needs, as well as the mock.patch string. $ python3 -m flake.mock ['return value for', 'read of Popen', "with ['a', 'b', 'c']"] ['return value for', 'read of Popen', "with ['d', 'e', 'f']"]

>>15516 >It tries to run the function that will download the current problem. I think it just returns the function python >>> def call(func): ... try: ... return func ... except: ... print("can't be called") ... >>> call(lambda x: x) <function <lambda> at 0xb691cc48> >>> def call(func): ... try: ... func() ... except: ... print("can't be called") ... >>> call(lambda x: x) can't be called

>>15525 Yeah, i meant to put the parentheses after it. That's why i said "see >>15512" I posted "s/return function/\0()/" right after the original post cause i noticed i was missing them.

Is there an elegant way to merge two dictionaries as follows? python DICT_1 = { "DOMAIN_1": [ "255.255.255.255" ], "DOMAIN_2": [ "16.16.16.16", "24.24.24.24", "32.32.32.32" ], "DOMAIN_3": [ "2.2.2.2", "4.4.4.4" ] } DICT_2 = { "DOMAIN_3": [ "8.8.8.8" ] "DOMAIN_4": [ "128.128.128.128" ] } RESULT = { # DICT_1 + DICT_2 "DOMAIN_1": [ "255.255.255.255" ], "DOMAIN_2": [ "16.16.16.16", "24.24.24.24", "32.32.32.32" ], "DOMAIN_3": [ "2.2.2.2", "4.4.4.4", "8.8.8.8" ], "DOMAIN_4": [ "128.128.128.128" ] }

>>15549 You have not specified a merging procedure for values. Since they are lists rather than sets, presumably you want all items preserved. >>> import collections >>> import functools >>> import itertools >>> import operator >>> def merge (*dicts): ... return functools.reduce (lambda acc, item: acc [item [0]].extend (item [1]) or acc, itertools.chain.from_iterable (map (operator.methodcaller ("items"), dicts)), collections.defaultdict (list)) ... >>> d = merge (DICT_1, DICT_2) >>> import pprint >>> pprint.pprint (d) {'DOMAIN_1': ['255.255.255.255'], 'DOMAIN_2': ['16.16.16.16', '24.24.24.24', '32.32.32.32'], 'DOMAIN_3': ['2.2.2.2', '4.4.4.4', '8.8.8.8'], 'DOMAIN_4': ['128.128.128.128']} >>>

>>15556 Your presumption is correct. Thanks for the help.

>>15528 Oh. What about this? python class DontDownloadAgain(Exception): def __init__(self, choice): self.choice = choice def retry(func): while True: try: func() except: choice = input('--> ') if choice == '1': continue else: raise DontDownloadAgain(choice) try: retry(lambda e: e) except DontDownloadAgain as e: if e.choice == '2': print('exit this problem. download the next') else: print('cancel') (I didn´t tested it)

Which is a good text editor and where should I run my script I know that there are lots of text editors out there but I dont like one that looks scuffed and not pycharm

>Python 3 Thread >share projects you have been working on A stamp painter that knows how to place a stream of items consisting of an image, a location and a rotation: class Paint: @ staticmethod def place (ctx, alpha, dx, dy, rot, surf): w = surf.get_width () h = surf.get_height () ctx.save () ctx.translate (dx, dy) ctx.rotate (rot) ctx.translate (-w / 2, -h / 2) ctx.set_source_surface (surf, 0, 0) ctx.paint_with_alpha (alpha) ctx.restore () @ staticmethod def stream (ctx, alpha, items): place = Paint.place for item in items: place (ctx, alpha, * item) class Items: def items (self, width, height): pass class FreeOverStamps (FreeOverAlpha): def __init__ (self, items): self.items = items def work (self, ctx, width, height, alpha): items = self.items.items (width, height) psta.Paint.stream (ctx, alpha, items) A producer of a stream of locations that will form a path. The Hilbert curve from >>15426 can be reused. class Points: def points (self): pass # points width height close # fresh lazy class Hilbert (Points): def points (self): hil = dtes.TestHilbert () cfg = hil.defaultConfig () size = cfg ["size"] cfg ["level"] = 3 pos = hil.makePoints (cfg) return { "points": pos, "width" : size, "height": size, "close" : cfg ["closepath"], } Angles of travel along a path are computed by first determining absolute orientations for segments, then rotations between adjoining segment orientations. These rotations and a starting angle are then used to build the absolute angles of travel, thus avoiding potentially unpleasant angle jumps like 270->0. Segments are then assigned starting, midpoint and ending angles. The handling of closed paths is for a future looping demo. class Angles: TURNS = {"P": 90, "S": 0, "N": -90} def __init__ (self, rotation_order): self.rotation_order = rotation_order def segment_to_dir (self, p1, p2): p1x, p1y = p1 p2x, p2y = p2 if p1x == p2x: if p1y == p2y: raise ValueError ("{} {} {}".format ( "same", p1, p2)) return "D" if p1y < p2y else "U" else: if p1y != p2y: raise ValueError ("{} {} {}".format ( "diag", p1, p2)) return "R" if p1x < p2x else "L" # straight positive negative def dirs_to_turn_build (self): if hasattr (self, "dirs_to_turn_storage"): return dirs = self.rotation_order out = { (c, c): "S" for c in dirs } size = len (dirs) for k in range (size): prev = dirs [k - 1] now = dirs [k] next = dirs [(k + 1) % size] out [(now, prev)] = "N" out [(now, next)] = "P" # print (out) self.dirs_to_turn_storage = out def dirs_to_turn_cached (self, d1, d2): return self.dirs_to_turn_storage [(d1, d2)] def initial_angle (self, dir): return self.rotation_order.index (dir) * 90 def turns_to_angles (self, firstdir, turns): adv = Angles.TURNS return functools.reduce ( lambda acc, item: acc.append (acc [-1] + adv [item]) or acc, turns, [self.initial_angle (firstdir)]) def angles_to_triples (self, angles, close, lastturn): it = iter (angles) first = next (it) def step (acc, item): out, start, seg = acc end = (seg + item) / 2 out.append ((start, seg, end)) return (out, end, item) acc = functools.reduce ( step, it, ([], first, first)) out, start, seg = acc if close: a1, a2, a3 = out [0] a1 = (a2 + a2 - Angles.TURNS [lastturn]) / 2 out [0] = (a1, a2, a3) else: out.append ((start, seg, seg)) return out def points_to_triples (self, points, close): starmap = itertools.starmap pairs = ither.pairs segdirs = list ( starmap (self.segment_to_dir, pairs (points))) # print (segdirs) if close: segdirs.append (segdirs [0]) self.dirs_to_turn_build () turns = list ( starmap (self.dirs_to_turn_cached, pairs (segdirs))) # print (turns) lastturn = turns [-1] if close else None segangles = self.turns_to_angles ( segdirs [0], turns) # print (segangles) seg3 = self.angles_to_triples (segangles, close, lastturn) # print (seg3) return seg3 Interpolation of pointwise angles along a segment is exported to a separate entity that can control the tightness of cornering: class Cornering: def inter (self, a1, a2, a3, time, steps): pass class Exponent (Cornering): def __init__ (self, exponent): self.exponent = exponent def inter (self, a1, a2, a3, time, steps): half = steps / 2 expo = self.exponent return ( a1 + (a2 - a1) * (1 - (1 - time / half) ** expo) if time < half else a2 + (a3 - a2) * ((time - half) / half) ** expo ) class Linear (Exponent): def __init__ (self): Exponent.__init__ (self, 1) A particular image knows its natural orientation and how to load itself per-frame. class Surface: def rotation_order (self): pass def surface (self, time, steps): pass class Alis (Surface): def __init__ (self, index): self.index = index def rotation_order (self): return "LURD" def surface (self, time, steps): ids = ("alis", "myau", "odin", "noah") pick = self.index src = "gif/{}-frame{}.png".format (ids [pick], time % 4) return cairo.ImageSurface.create_from_png (src) A path walker takes a stream of waypoints, a frame-aware image surface and an angle interpolator to produce a subpath of items for the stamp painter. A list of subpaths forms a path. class PathWalk (Items): DEFAULT_ANGLE = Linear () DEFAULT_SURFACE = Heart () def __init__ (self, steps, paths): self.items_steps = steps self.items_paths = paths def items_path (self, width, height, path): path_points = path ["points"] path_angle = path.get ("angle" , PathWalk.DEFAULT_ANGLE ) path_surface = path.get ("surface", PathWalk.DEFAULT_SURFACE) steps = self.items_steps time = self.time surf = path_surface.surface (time, steps) ret = path_points.points () pos = ret ["points"] sizew = ret ["width" ] sizeh = ret ["height"] close = ret ["close" ] if not isinstance (pos, list): pos = list (pos) if close: pos.append (pos [0]) # print (pos) starmap = itertools.starmap pairs = ither.pairs def f1 (a, b): return a + (b - a) * time / steps xy = starmap ( lambda p1, p2: (f1 (p1 [0], p2 [0]), f1 (p1 [1], p2 [1])), pairs (starmap ( lambda x, y: (x * width / sizew, y * height / sizeh), pos))) angles = Angles (path_surface.rotation_order ()) seg3 = angles.points_to_triples (pos, close) f3 = path_angle.inter radians = math.radians degrees = starmap ( lambda a1, a2, a3: f3 (a1, a2, a3, time, steps), seg3) return list (map ( lambda p, a: (p [0], p [1], radians (a), surf), xy, degrees)) def items (self, width, height): self.time = 0 work = self.items_path return itertools.chain.from_iterable (map ( lambda p: work (width, height, p), self.items_paths)) A stamp painter for a Hilbert path with quadratic cornering: class FreeOver_hilbert2 (FreeOverStamps): def __init__ (self): FreeOverStamps.__init__ (self, psta.PathWalk (12, [{ "points" : psta.Hilbert (), "surface": psta.Alis (1), "angle" : psta.Exponent (2)}])) As a result, here is Myau from Phantasy Star running the Hilbert curve all over SICP.

>>15564 Well, your "retry" function will retry until there's an error and the user doesn't try again, but i guess you can find that out by testing. I'm not sure what conceptual question you're trying to ask here. >>15591 >Which is a good text editor vim is a good text editor, though there may be others. >where should I run my script i usually run it in whatever directory it's stored in.

>Python 3 Thread >share projects you have been working on A class that can generate a list of points by following a series of steps on a cell grid: class PointsSpec: DIRECTIONS = { "U": ( 0, -1), "D": ( 0, 1), "L": (-1, 0), "R": ( 1, 0), } @ staticmethod def celldirs (dirs, origin, cellsize): out = [origin] delta = PointsSpec.DIRECTIONS def step (acc, item): x , y = acc [ -1] dx, dy = delta [item] acc.append ((x + dx * cellsize, y + dy * cellsize)) return acc functools.reduce (step, dirs, out) return out Shapes of letters are formed by series of grid steps, with runs represented by repeat counts: class Letters: CELLSIZE = 32 ROWS = 9 COLUMNS = 5 PAD = CELLSIZE / 2 SPECSTR = "([UDLR])([0-9]*)" SPECPAT = re.compile (SPECSTR) DEFAULT_CONFIG = { "cell": CELLSIZE, "pad" : PAD, } A = [{ "origin" : (0, 0), "dirspec": "D9RU4R3D4RU9L4"}, { "origin" : (1, 1), "dirspec": "D3R3U3L2", }] I = [{ "origin" : (0, 1), "dirspec": "DRD7LDR3ULU7RUL2", }] L = [{ "origin" : (0, 0), "dirspec": "D9R5UL4U8", }] N = [{ "origin" : (0, 0), "dirspec": "D9RU8R3D8RU9L4", }] To unpack the runs, a regular expression identifies the repeat counts and each step is multiplied by its count: class Letters: @ staticmethod def unpack (dirspec): def fun (mobj): ch, rep = mobj.groups () rep = int (rep) if rep else 1 return ch * rep return ''.join (map ( fun, Letters.SPECPAT.finditer (dirspec))) With these a subpath of a letter can be made into a Points >>15593 object's output: class Letters: @ staticmethod def points_sub (lettersub, *, config = None): row, col = lettersub ["origin" ] spec = lettersub ["dirspec"] if config is None: config = Letters.DEFAULT_CONFIG cell = config ["cell"] pad = config ["pad" ] return { "points": PointsSpec.celldirs ( Letters.unpack (spec), (col * cell + pad, row * cell + pad), cell), "width" : Letters.COLUMNS * cell + 2 * pad, "height": Letters.ROWS * cell + 2 * pad, "close" : True, } A list of subpaths forms the path of a full letter: class Letters: @ staticmethod def points (letter, *, config = None): return [ PointsProxy ( lambda sub: Letters.points_sub (sub, config = config), (sub,)) for sub in letter ] class PointsProxy (Points): def __init__ (self, producer_fun, producer_args): self.producer_fun = producer_fun self.producer_args = producer_args def points (self): return self.producer_fun (* self.producer_args) The subpaths of a letter are loops, so the closed path support from Angles is needed. For a loop the starting point is appended to the point list to form the closing segment with the last point. The first segment is appended to the segment list to get the turn from the closing segment to it. This final turn is run backward from the first segment to adjust the first segment's starting angle. This will match the closing segment's ending angle modulo 360. To work towards the goal of combining letters to form a string, Points can have transformations applied to them: class Transform: def transform (self, points): return points class TransformedPoints (Points): @ staticmethod def apply (points, transforms): return functools.reduce ( lambda acc, item: item.transform (acc), transforms, points) def __init__ (self, points, transforms): self.subpoints = points self.transforms = transforms def points (self): return TransformedPoints.apply ( self.subpoints.points (), self.transforms) Of interest to letters are transforms that enlarge the containing area and that translate the point list in a letter grid: class FieldFactor (Transform): def __init__ (self, fx, fy): self.fx = fx self.fy = fy def transform (self, points): points ["width" ] *= self.fx points ["height"] *= self.fy return points class GridMove (Transform): def __init__ (self, row, col): self.row = row self.col = col def transform (self, points): pos = points ["points"] w = points ["width" ] h = points ["height"] row = self.row col = self.col points ["points"] = itertools.starmap ( lambda x, y: (x + col * w, y + row * h), pos) return points These can be used to build a combinator that produces a monospaced row of its input paths: class Combinator: @ staticmethod def monorow (pogroups): count = len (pogroups) ff = FieldFactor (count, 1) flatten = itertools.chain.from_iterable starmap = itertools.starmap return list (flatten (starmap ( lambda k, group: map ( lambda p, move = GridMove (0, k): TransformedPoints (p, [move, ff]), group), enumerate (pogroups)))) The combinator can be used to transform text into letter paths, then arrange them into a monospaced row: class Letters: @ staticmethod def text_to_monorow (text, *, config = None): letters = [ getattr (Letters, ch) for ch in text ] counts = [ len (let) for let in letters ] groups = [ Letters.points (let, config = config) for let in letters ] row = Combinator.monorow (groups) return { "counts": counts, "row" : row, } A demo that pairs letters with per-frame image surfaces and hands the result off to the path walker: class Letters: @ staticmethod def demo (): text = "LAIN" ret = Letters.text_to_monorow (text) row = ret ["row" ] counts = ret ["counts"] flatten = itertools.chain.from_iterable starmap = itertools.starmap repeat = itertools.repeat surf = flatten (starmap ( lambda k, len: repeat (Alis (k % 4), len), enumerate (counts))) return list (map ( lambda p, s: {"points": p, "surface": s}, row, surf)) class FreeOver_stamps2 (FreeOverStamps): def __init__ (self): FreeOverStamps.__init__ (self, psta.PathWalk (12, psta.Letters.demo ())) As a result, here is the Phantasy Star team walking the Lain.