100inoneyear

Installing (build) Matplotlib In Mac Osx Lion

2023-09-27T12:02:00.001-07:00

I installed pynum and scipy (on osx Lion with python 2.7), but when I tried to build matplotlib git clone https://github.com/matplotlib/matplotlib.git cd matplotlib python setup.p

Solution 1:

I always find this process more painful than it should be, but I've done it a few times now and I believe that these steps should get you set up:

Get Xcode 4.3.2, it's required for some of the later steps.
Download the latest version of python for OSX from python.org
Grab the Scipy superpack.
Uninstall any previous versions of numpy/matplotlib/scipy that you currently have. That includes doing cd /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/ and moving any numpy/matplotlib/scipy directories or eggs into a temp directory.
cd ~/Downloads(or wherever you downloaded the superpack script to) and run sh install_superpack.sh. Answer no to the question are you installing from a repository cloned to this machine or you'll be confused about why the script keeps failing.

That should be it! You should now be able to boot up the python console and import numpy, scipy, matplotlib.

Feedparser - Keyerror: 'fullcount'

2023-09-27T11:08:00.001-07:00

I have tried to follow this guide. It is about making a physical gmail notifier. When I entered the same code it found an error: Traceback (most recent call last): File 'C:/Pytho

Solution 1:

Just took a look at that from a REPL. The code will work as-is for me. However, I was able to reproduce your error by entering an incorrect password.

This is what feedparser.parse()['feed'] looks like if you fail to authenticate:

>>> feedparser.parse(PROTO + USERNAME + ":" + INCORRECT_PASSWORD + "@" + SERVER + PATH)['feed']
{'summary': u'<h1>Unauthorized</h1>\n<h2>Error 401</h2>'}
>>>

This is what it should look like if you do authenticate properly:

>>> import pprint
>>> pprint.pprint(feedparser.parse(PROTO + USERNAME + ":" + PASSWORD + "@" + SERVER + PATH)['feed'])
{'fullcount': u'0',
 'link': u'http://mail.google.com/mail',
 'links': [{'href': u'http://mail.google.com/mail',
            'rel': u'alternate',
            'type': u'text/html'}],
 'subtitle': u'New messages in your Gmail Inbox',
 'subtitle_detail': {'base': u'https://mail.google.com/mail/feed/atom',
                     'language': None,
                     'type': u'text/plain',
                     'value': u'New messages in your Gmail Inbox'},
 'title': u'Gmail - Inbox for xxxxxxx@gmail.com',
 'title_detail': {'base': u'https://mail.google.com/mail/feed/atom',
                  'language': None,
                  'type': u'text/plain',
                  'value': u'Gmail - Inbox for xxxxxxx@gmail.com'},
 'updated': u'2013-03-01T20:11:03Z',
 'updated_parsed': time.struct_time(tm_year=2013, tm_mon=3, tm_mday=1, tm_hour=20, tm_min=11, tm_sec=3, tm_wday=4, tm_yday=60, tm_isdst=0)}
>>>

You should print out the result of feedparser.parse() to check if this is indeed the problem you have, but I suspect that simply making sure your username/password are correct will fix your problem

Ipython Importerror: No Module Named Display

2023-09-27T10:09:00.001-07:00

Trying to run python script but can't find module IPython.display.. Seems like it would be something simple but I can't find an answer anywhere! /vagrant/dreamify.py in

Solution 1:

According to the IPython's repository, the IPython.display module was added in commit 5462c7e, and it was released with IPython 0.13.

I think you have an older version of IPython.

You can check the IPython version with ipython --version.

It can be upgraded with pip install --upgrade IPython.

Solution 2:

In Pycharm import module ipython. In Pycharm go to preferences and then project interpreter. Click on the "+" sign then search ipython and install package.

Tkinter Button Not Changing Global Variable

2023-09-27T09:07:00.001-07:00

I have started learning python gui.Button function call doesn't update the global variable. Currently I am facing problem in below code. from tkinter import * root=Tk() root.titl

Solution 1:

Try this:

from tkinter import *
root=Tk()  
root.title("learning")    
x=""def change() :   
    global x
    x="program"
    print(x)

def prt():
    global x
    print(x)

Button(root,text="Click me", command=change).pack()
Button(root,text="Print x", command=prt).pack()

root.mainloop()     
print(x)

Try to do this:

Click 'Print x' first - This should print an empty space
Then click on Click Me - x changes to program
And then again click on 'Print x' - this should print program

Another error - button1 is None as you have used .pack on the same line. So if you want to use button1 in your code ahead, use it as:

button1 = Button(root, text='Click me', command=change)
button1.pack()

Solution 2:

Try this:

from tkinter import *
    
    defchange():
        globalstrstr="program"print(str) # print function should be placed here
    
    root=Tk()
    root.title("learning")
    root.geometry("300x300")
    str=""
    button1=Button(root,text="Click me", command=change).pack()
    root.mainloop()

when the button is clicked, immediately the function change() will be called so, the print function should be placed in change()

Python: When Writing To A Large File, Keep The File Open Or To Open It And Append To It As Needed?

2023-09-27T06:11:00.001-07:00

I am wondering how to best handle writing to a big file in python. My python code loops many times running an external program (ancient Fortran with a weird input file format), re

Solution 1:

Opening and closing the files definitely has a cost. However if your legacy program takes one or more second to respond you propably won't notice.

deffunc1():
    for x inrange(1000):
        x = str(x)
        withopen("test1.txt", "a") as k:
            k.write(x)

1 loops, best of 3: 2.47 s per loop

deffunc2():
    withopen("test2.txt", "a") as k:
        for x inrange(1000):
            x = str(x)
            k.write(x)

100 loops, best of 3: 6.66 ms per loop

However if your file get's really big it becomes slower: (800+mb)

deffunc3(file):
    for x inrange(10):
        x = str(x)
        withopen(file, "a") as k:
            k.write(x)

12kb file:

10 loops, best of 3: 33.4 ms per loop

800mb+ file:

1 loops, best of 3: 24.5 s per loop

Keeping the file open will mainly cost you memory.

I would suggest using SQlite to store your data.

Render Pandas Dataframe To Html Table With Row Seperators

2023-09-27T04:48:00.001-07:00

I render a pandas multi-index into an html table. When rendering, after I passed a specific index X I want to add a separator line into the table. I can do this manually by editin

Solution 1:

You can groupby index and loop through it while adding the separator:

df = pd.DataFrame({"col1":list("AABBCCA"),"unit":[1,2,1,3,4,4,6]})

for _, data in df.groupby("col1"):
    print (data.to_html())
    print ('<td colspan="20" class="divider"><hr /></td>')

Result:

Combining Lists With Overlapping Elements

2023-09-27T03:47:00.001-07:00

I have a collection of lists, some of which have overlapping elements: coll = [['aaaa', 'aaab', 'abaa'], ['bbbb', 'bbbb'], ['aaaa', 'bbbb'], ['dddd', 'ddd

Solution 1:

Here's a way to do it (assuming you want unique elements over the overlapping results):

defover(coll):
     print('Input is:\n', coll)
     # gather the lists that do overlap 
     overlapping = [x for x in coll ifany(x_element in [y for k in coll if k != x for y in k] for x_element in x)] 
     # flatten and get unique 
     overlapping = sorted(list(set([z for x in overlapping for z in x]))) 
     # get the rest
     non_overlapping = [x for x in coll ifall(y notin overlapping for y in x)] 
     # use the line bellow only if merged non-overlapping elements are desired# non_overlapping = sorted([y for x in non_overlapping for y in x]) print('Output is"\n',[overlapping, non_overlapping])

coll = [['aaaa', 'aaab', 'abaa'],
        ['bbbb', 'bbbb'], 
        ['aaaa', 'bbbb'], 
        ['dddd', 'dddd'],
        ['bbbb', 'bbbb', 'cccc','aaaa']]
over(coll)
coll = [['aaaa', 'aaaa'], ['bbbb', 'bbbb']]
over(coll)

output:

$ python3 over.py                                                                                                                                                              -- NORMAL --
Input is:
 [['aaaa', 'aaab', 'abaa'], ['bbbb', 'bbbb'], ['aaaa', 'bbbb'], ['dddd', 'dddd'], ['bbbb', 'bbbb', 'cccc', 'aaaa']]
Output is"
 [['aaaa', 'aaab', 'abaa', 'bbbb', 'cccc'], [['dddd', 'dddd']]]
Input is:
 [['aaaa', 'aaaa'], ['bbbb', 'bbbb']]
Output is"
 [[], [['aaaa', 'aaaa'], ['bbbb', 'bbbb']]]

Solution 2:

You can do this with sets using a successive merging approach:

coll = [['aaaa', 'aaab', 'abaa'],
        ['bbbb', 'bbbb'], 
        ['aaaa', 'bbbb'], 
        ['dddd', 'dddd'],
        ['bbbb', 'bbbb', 'cccc','aaaa'],
        ['eeee','eeef','gggg','gggi'],
        ['gggg','hhhh','iiii']]

pooled = [set(subList) for subList in coll]
merging = True
while merging:
    merging=False
    for i,groupinenumerate(pooled):
        merged = next((g for g in pooled[i+1:] if g.intersection(group)),None)
        ifnot merged: continuegroup.update(merged)
        pooled.remove(merged)
        merging = True

print(pooled)
# [{'aaaa', 'abaa', 'aaab', 'cccc', 'bbbb'}, {'dddd'}, {'gggg', 'eeef', 'eeee', 'hhhh', 'gggi', 'iiii'}]

Solution 3:

Working on a suggestion from alkasm in the comments, I used networkx:

import networkx as nx

coll = [['aaaa', 'aaab', 'abaa'],
        ['bbbb', 'bbbb'], 
        ['aaaa', 'bbbb'], 
        ['dddd', 'dddd'],
        ['bbbb', 'bbbb', 'cccc','aaaa'],
        ['eeee','eeef','gggg','gggi'],
        ['gggg','hhhh','iiii']]

edges = []
for i inrange(len(coll)):
    a = coll[i]
    for j inrange(len(coll)):
        if i != j:
            b = coll[j]
            ifset(a).intersection(set(b)):
                edges.append((i,j))

G = nx.Graph()
G.add_nodes_from(range(len(coll)))
G.add_edges_from(edges)

for c in nx.connected_components(G):
    combined_lists = [coll[i] for i in c]
    flat_list = [item for sublist in combined_lists for item in sublist]
    print(set(flat_list))

Output:

{'cccc', 'bbbb', 'aaab', 'aaaa', 'abaa'}
{'dddd'}
{'eeef', 'eeee', 'hhhh', 'gggg', 'gggi', 'iiii'}

Undoubtedly this can be optimized but it seems to solve my problem for now.

How To Use Python Map Function To Generate The First 100 Powers Of Two Starting At 0 And Exclusive Of 100?

2023-09-26T23:49:00.001-07:00

def map_twos(): def pwr(x): return 2**x map(pwr, range(0, 100)) print pwr.get[2] return pwr What is wrong. I don't know how do I use map function to get t

Solution 1:

pwr is a function. Even after you map it over a list, pwr is still a function. map(pwr, range(0, 100)) is the list you want. You can store it in a variable:

powers = map(pwr, range(0, 100))
print powers[2]
return powers

Solution 2:

map does not modify the existing list or function, but instead generates an iterator over the result (in Python 3). That means you need to store the result of map and eventually convert it to a list of you need to get the n-th element (in Python 3).

pwrs = map(lambda n: 1 << n, xrange(100))
print pwrs[2]

From the use of the print statement I figured you were using Python 2, so I left out the explicit conversion to a list and used xrange which returns an iterator (leading to better performance). In Python 3 xrange does not exist anymore however and range always returns an iterator. Also notice how I left out the lower boundary on the range function, because it defaults to 0. Instead of predefining a function I used a lambda here, which in turn uses a bitwise leftshift , because it is usually faster than using the ** (power) operator.

What Is The Difference Between .semaphore() And .boundedsemaphore()?

2023-09-26T22:49:00.001-07:00

I know that threading.Lock() is equal to threading.Semaphore(1). Is also threading.Lock() equal to threading.BoundedSemaphore(1) ? And newly I saw threading.BoundedSemaphore(), wha

Solution 1:

A Semaphore can be released more times than it's acquired, and that will raise its counter above the starting value. A BoundedSemaphorecan't be raised above the starting value.

from threading import Semaphore, BoundedSemaphore

# Usually, you create a Semaphore that will allow a certain number of threads# into a section of code. This one starts at 5.
s1 = Semaphore(5)

# When you want to enter the section of code, you acquire it first.# That lowers it to 4. (Four more threads could enter this section.)
s1.acquire()

# Then you do whatever sensitive thing needed to be restricted to five threads.# When you're finished, you release the semaphore, and it goes back to 5.
s1.release()


# That's all fine, but you can also release it without acquiring it first.
s1.release()

# The counter is now 6! That might make sense in some situations, but not in most.print(s1._value)  # => 6# If that doesn't make sense in your situation, use a BoundedSemaphore.

s2 = BoundedSemaphore(5)  # Start at 5.

s2.acquire()  # Lower to 4.

s2.release()  # Go back to 5.try:
    s2.release()  # Try to raise to 6, above starting value.except ValueError:
    print('As expected, it complained.')

Solution 2:

The threading module provides the simple Semaphore class.

A Semaphore provides a non-bounded counter which allows you to call release() any number of times for incrementing.

However, to avoid programming errors, it’s usually a correct choice to use BoundedSemaphore , which raises an error if a release() call tries to increase the counter beyond its maximum size.

EDIT

A semaphore has an internal counter rather than a lock flag (in case of Locks), and it only blocks if more than a given number of threads have attempted to hold the semaphore. Depending on how the semaphore is initialized, this allows multiple threads to access the same code section simultaneously.

Why Are Explicit Calls To Magic Methods Slower Than "sugared" Syntax?

2023-09-26T19:43:00.001-07:00

I was messing around with a small custom data object that needs to be hashable, comparable, and fast, when I ran into an odd-looking set of timing results. Some of the comparisons

Solution 1:

Two reasons:

The API lookups look at the type only. They don't look at self.foo.__hash__, they look for type(self.foo).__hash__. That's one less dictionary to look in.
The C slot lookup is faster than the pure-Python attribute lookup (which will use __getattribute__); instead looking up the method objects (including the descriptor binding) is done entirely in C, bypassing __getattribute__.

So you'd have to cache the type(self._foo).__hash__ lookup locally, and even then the call would not be as fast as from C code. Just stick to the standard library functions if speed is at a premium.

Another reason to avoid calling the magic methods directly is that the comparison operators do more than just call one magic method; the methods have reflected versions too; for x < y, if x.__lt__ isn't defined or x.__lt__(y) returns the NotImplemented singleton, y.__gt__(x) is consulted as well.

Pyusb On Windows 8.1 - No Backend Available - How To Install Libusb?

2023-09-26T17:38:00.001-07:00

Working on pyinstaller 3.1, python 2.7.9, and tkinter. Try to use pyusb instead of pyserial, but no backend available. Looking into Pyusb on windows - no backend available, PyUSB V

Solution 1:

I downloaded latest libusb windows binaries from here: http://sourceforge.net/projects/libusb/files/libusb-1.0/libusb-1.0.20/libusb-1.0.20.7z/download then I copied libusb1.dll according to this: when you are on 64-bit Windows, MS64\dll\libusb-1.0.dll must be copied into C:\Windows\System32 and (for running 32-bit applications that use libusb) MS32\dll\libusb-1.0.dll must be copied into C:\Windows\SysWOW64. and then I copied libusb1.lib from binaries to lib dir in python directory and now pyusb works for me. hope it helps.

Save Optimization Of Hyperparameter In A Convolutional Neural Net

2023-09-26T15:30:00.001-07:00

I am facing a problem regarding saving the training process of the hyperparameter in my Convolutional Neural Net. I have read couple of blog posts, but somehow I am unable to do th

Solution 1:

I do not have the exact answer to that question, however there is here a "trick" that might do it.

It proposes to print the content of every trial tested, at the end of the code. Maybe you can save the "trials" object as well in a pickle or something, so that you can parse and check for yourself later. I do have the exact same question and i'm very very suprise that there is no "easy" solution as with the keras callbacks to save the best trained model.

I also suggest you to change the title of your questiopn, adding at least "Hyperopt", "callbacks" and "saving model" key words. With more attention maybe our question will be answered :)

How To Create Page Counter With Shuffled Links

2023-09-26T13:22:00.001-07:00

I am wanting to create page count down for shuffled links so easily informed of where my script is actually progressing. I try: def page_counter(): for x in range(1000):

Solution 1:

try this

Replace

links = dict((next(count) + 1, e) for e in elements)
desc_links = sorted(links.items(), key=operator.itemgetter(1))

for link in desc_links:
    driver.get(link[1])
    print(f'At Page: {link[0]}')

With

import collections
links = dict((next(count) + 1, e) for e in elements)

desc_links = collections.OrderedDict(sorted(links.items(), reverse=True))

for key, value in desc_links.items():
    driver.get(value)
    print('At Page: ' + str(key))

How To Correlate Scalar Values Of Two Pandas Dataframes

2023-09-26T12:13:00.001-07:00

How do I correlate two pandas dataframes, find a single r value for all values? I don't want to correlate columns or rows, but all scalar values. One dataframe is the x axis, and t

Solution 1:

I've simulated a setup that I think mimics yours--three dataframes with countries across rows and years across columns. I then concatenate the different sets of data into a single dataframe. And show how to compute the correlation between them. Let me know if something about this example doesn't match your setup.

import pandas as pdset1= pd.DataFrame({1980:[4, 11, 0], 1981:[5, 10, 2], 1982:[0, 3, 1]},
    index=pd.Index(['USA', 'UK', 'Iran'], name='country'))
set1.columns.name = 'year'
set1

year1980  1981  1982countryUSA450UK11103Iran021

set2 = pd.DataFrame({1981:[2, 1, 10], 1982:[15, 1, 12], 1983:[10, 13, 1]},
    index=pd.Index(['USA', 'UK', 'Turkey'], name='country'))
set2.columns.name = 'year'
set2

year1981  1982  1983countryUSA21510UK1113Turkey10121

Notice that, like your setup, some countries/years are not present in different datasets.

set3 = pd.DataFrame({1980:[12, 11, 4], 1982:[9, 8, 11]},
    index=pd.Index(['USA', 'UK', 'Turkey'], name='country'))
set3.columns.name = 'year'

We can turns these into multi-indexed series by stacking along year and then concatenate these across columns using pd.concat.

df = pd.concat([set1.stack('year'), set2.stack('year'), set3.stack('year')],
    keys=['set1', 'set2', 'set3'], names=['set'], axis=1)
df

setset1set2set3countryyearIran1980   0.0NaNNaN1981   2.0NaNNaN1982   1.0NaNNaNTurkey1980   NaNNaN4.01981   NaN10.0NaN1982   NaN12.011.01983   NaN1.0NaNUK1980  11.0NaN11.01981  10.01.0NaN1982   3.01.08.01983   NaN13.0NaNUSA1980   4.0NaN12.01981   5.02.0NaN1982   0.015.09.01983   NaN10.0NaN

And we can compute a 3x3 correlation matrix across the three different sets.

df.corr()

set       set1      set2      set3
set                               
set1  1.000000 -0.723632  0.509902
set2 -0.723632  1.000000  0.606891
set3  0.509902  0.606891  1.000000

Solution 2:

Here's what I did, but it's still not as slick as if I had found a built-in pandas feature or package.

Because I ultimately wanted to do this with more than two tables, I put the tables (dataframes) into a dictionary.

Then, I changed each table into a one-column table that has a MultiIndex representing the original column names and index values. The field values are the original column values strung end to end.

Then, I merged these new tables into one full outer join on the MultiIndex. Now I can correlate any two of the original tables by correlating their respective columns in the final table.

import pandas as pd

gvtx_eiu_df = pd.read_csv('gvtx_eiu.csv',index_col=0,
                          skip_blank_lines=False)
gvtx_eiu_df.columns.name = 'year'
polpartix_eiu_df = pd.read_csv('polpartix_eiu.csv',index_col=0,
                               skip_blank_lines=False)
polpartix_eiu_df.columns.name = 'year'
clean_elec_idea_df = pd.read_csv('clean_elec_idea.csv', index_col=0,
                                 skip_blank_lines=False)
clean_elec_idea_df.columns.name = 'year'

test_table_dict = {'gvtx_eiu': gvtx_eiu_df,
                   'polpartix_eiu': polpartix_eiu_df,
                   'clean_elec_idea': clean_elec_idea_df}
'''
# Updated to not use this anymore. Using stack now, thanks to @jtorca. So it# fits more neatly into one function.# Serialize df columns into MultiIndex df, index=(year, country), one column
def df_to_multidx_df(df: pd.DataFrame, cols_idx1_name: str = 'Previous Columns',
                     idx_idx2_name: str = 'Previous Index',
                     val_col_name: str = 'Values') -> pd.DataFrame:
    #Takes 2d dataframe (df) with a single-level index and one or more#single-level columns. All df values must be the same type.#Parameters:#    df: 2d dataframe with single-level index and one or more#        single-level columns. All df values must be the same type.#    cols_idx1_name: 1st index title for returned dataframe; index is df#        column names.#    idx_idx2_name: 2nd index title for returned dataframe; index is df#        index.#Returns:#    a 2d dataframe with a MultiIndex constructed of table_df column#    names and index values. Has a single column with field values that are#    all df columns strung end to end.# Create MultiIndex from product of index values and column names.
    mult_idx = pd.MultiIndex.from_product([df.columns, df.index],
                                          names=[cols_idx1_name, idx_idx2_name])
    # 1D list of table values in same order as MultiIndex.
    val_list = [val for col in df for val in df[col]]
    
    return pd.DataFrame(val_list, index=mult_idx, columns=[val_col_name])
'''

def df_dict_to_multidx_df(df_dict: dict) -> pd.DataFrame:
#     , cols_idx1_name: str = 'idx1',#     idx_idx2_name: str = 'idx2') -> pd.DataFrame:
    '''Converts a dictionary (df_dict) of 2d dataframes, each with single-level
    indices and columns, into a 2d dataframe (multidx_df) with each column
    containing the the values of one of df_dict's dataframes. The index of
    multidx_df is a MultiIndex of the input dataframes' column names and index
    values. Dataframes are joined in full outer join on the MultiIndex.
        NOTE: each input dataframe's index and columns row must be named
        beforehand in order to name the columns in the multiindex and join on it.
    Parameters:
        df_dict: dictionary of 2d dataframes, each with single-level
            indices and columns.
    Returns:
        multidx_df = MultiIndex dataframe.'''
    
    df_dict_copy = df_dict.copy()
        
    # Full outer join each table to multidx_df on MultiIndex.# Start with first indicator to have a left df to merge.
    first_key = next(iter(df_dict_copy))
    multidx_df = pd.DataFrame(df_dict_copy.pop(first_key).stack(),
                                     columns=[first_key])
    for key, df in df_dict_copy.items():
        df = pd.DataFrame(df.stack(), columns=[key])
        multidx_df = multidx_df.merge(right=df, how='outer',
                                     on=multidx_df.index.names[:2])

        # concat twice as fast as merge#         multidx_df = pd.concat([multidx_df, df], names=['indicator'], axis=1)
    
    return multidx_df

###Test Code

print(gvtx_eiu_df)

#               2006    2007   2008   2009   2010   2011   2012   2013   2014  \# country                                                                       # Afghanistan    NaN  0.0395  0.079  0.079  0.079  0.079  0.079  0.079  0.114   # Albania      0.507  0.5070  0.507  0.507  0.507  0.471  0.400  0.400  0.400   # Algeria      0.221  0.2210  0.221  0.221  0.221  0.221  0.221  0.221  0.221   # Angola       0.214  0.2680  0.321  0.321  0.321  0.321  0.321  0.321  0.321   # Argentina    0.500  0.5000  0.500  0.535  0.571  0.571  0.571  0.571  0.571   # ...            ...     ...    ...    ...    ...    ...    ...    ...    ...   # Venezuela    0.364  0.3960  0.429  0.411  0.393  0.393  0.429  0.429  0.429   # Vietnam      0.429  0.4290  0.429  0.429  0.429  0.429  0.393  0.393  0.393   # Yemen        0.271  0.2610  0.250  0.214  0.179  0.036  0.143  0.143  0.143   # Zambia       0.464  0.4640  0.464  0.500  0.536  0.500  0.536  0.536  0.536   # Zimbabwe     0.079  0.0790  0.079  0.104  0.129  0.129  0.129  0.129  0.129   #               2015   2016   2017   2018  # country                                  # Afghanistan  0.114  0.114  0.114  0.114  # Albania      0.436  0.436  0.471  0.471  # Algeria      0.221  0.221  0.221  0.221  # Angola       0.321  0.321  0.286  0.286  # Argentina    0.500  0.500  0.500  0.536  # ...            ...    ...    ...    ...  # Venezuela    0.393  0.250  0.286  0.179  # Vietnam      0.393  0.321  0.321  0.321  # Yemen        0.036    NaN    NaN    NaN  # Zambia       0.536  0.536  0.500  0.464  # Zimbabwe     0.200  0.200  0.200  0.200  # [164 rows x 13 columns]


test_serialized = df_to_multidx_df(df=gvtx_eiu_df, cols_idx1_name='Year',
                                   idx_idx2_name='Country',
                                   val_col_name='gvtx_eiu')
print(test_serialized)

#                       gvtx_eiu# Year Country                  # 2006 Afghanistan           NaN#      Albania             0.507#      Algeria             0.221#      Angola              0.214#      Argentina           0.500# ...                        ...# 2018 Venezuela           0.179#      Vietnam             0.321#      Yemen                 NaN#      Zambia              0.464#      Zimbabwe            0.200# [2132 rows x 1 columns]


test_multidx_df = table_dict_to_multidx_df(test_table_dict, 'Year', 'Country')

print(test_multidx_df)

#                       gvtx_eiu       polpartix_eiu  clean_elec_idea# Year Country                                                       # 2006 Afghanistan           NaN               0.222            0.475#      Albania             0.507               0.444            0.541#      Algeria             0.221               0.222            0.399#      Angola              0.214               0.111              NaN#      Argentina           0.500               0.556            0.778# ...                        ...                 ...              ...# 2017 Somalia               NaN                 NaN            0.394#      South Sudan           NaN                 NaN              NaN# 2018 Georgia               NaN                 NaN            0.605#      Somalia               NaN                 NaN              NaN#      South Sudan           NaN                 NaN              NaN# [6976 rows x 3 columns]

test_multidx_profile = ProfileReport(test_multidx_df, title='Test MultIdx Profile')

The output is exactly what I was going for, but in addition to wishing for a one- or two-statement solution, I'm not completely happy with iterating through an input dictionary of dataframes. I tried to make the input a dataframe of dataframes so I could apply(lambda) to save some memory I think, but no dice getting apply() to work right, and it's time to move on.

Tkinter Button In Class Can't Call Function

2023-09-26T10:11:00.001-07:00

Total noob, seriously and angrily struggling with Python... What I'm trying to do SHOULD be simple: Make a button. Connect that button go a function. Click button --> run funct

Solution 1:

If you want your functions to be designed outside of the class, but want to call it from a button defined in the class, the solution is to create a method in your class which takes the input values and then passes them to the external function.

This is called decoupling. Your function is decoupled from the implementation of the UI, and it means that you are free to completely change the implementation of the UI without changing the function, and vice versa. It also means that you can reuse the same function in many different programs without modification.

The overall structure of your code should look something like this:

# this is the external function, which could be in the same# file or imported from some other moduledefconvert(kg):
    pounds = kg * 2.2046return pounds

classExample(...):
    definitUI(self):
        ...
        self.entry_kg = Entry(...)
        btn_convert=Button(..., command=self.do_convert)
        ...

    defdo_convert(self):
        kg = float(self.entry_kg.get())
        result = convert(kg)
        print("%d kg = %d lb" % (kg, result))

Solution 2:

Here's a modified version of your code that works. The changes have been indicated with ALL CAPS line comments. I obviously misunderstood your question (which does say you could figure out how to make the convert() function part of the class. However, you mentioned you wanted the opposite of that, so I'm modified the code here accordingly.

Essentially the problem boils down to the convert() function needing to access a tkinter.Entry widget that's created somewhere else—inside your Example class in this case.

One way of doing that would be to save the widget in a global variable and access it through the variable name assigned to it. Many folks do that because it's easiest to thing to do, but as you should know, global variables are considered a bad practice and are generally something to be avoided.

There's a common way to avoid needing one with tkinter, which is sometimes called "The extra arguments trick". To use it all you need to do is create a short anonymous function with an argument that has a default value defined—which in this case will be the Entry widget you want passed to the now "wrapped" convert() function. This can be done using what's called a lambda expression. The revised code below and the comments in it show and describe how to do this:

from tkinter import *
from tkinter.ttk import Frame, Button, Style


defconvert(entry_widget):  # ADDED WIDGET ARGUMENT""" Some function outside class. """print("clicked")
    kg = entry_widget.get()  # REFERENCE ENTRY WIDGET PASSED AS ARGUMENTprint(kg)


classExample(Frame):
    def__init__(self):
        super().__init__()

        self.initUI()    # initiate the GUIdefinitUI(self):
        self.master.title("Weight Converter")
        self.pack(fill=BOTH, expand=True)

        frame_kg = Frame(self)   # frame for Kilograms
        frame_kg.pack(fill=X)

        lbl_kg = Label(frame_kg, text="Kilograms", width=16)
        lbl_kg.pack(side=LEFT, padx=5, pady=5)

        entry_kg = Entry(frame_kg)
        entry_kg.pack(fill=X, padx=(5, 30), expand=True)

        frame_btn = Frame(self)    # frame for buttons
        frame_btn.pack(fill=BOTH, expand=True, padx=20, pady=5)

        btn_convert=Button(frame_btn, text="Convert",
                        # DEFINE ANONYMOUS FUNCTION WITH DEFAULT ARGUMENT SO IT'S# AUTOMATICALLY PASSED TO THE TARGET FUNCTION.
                        command=lambda entry_obj=entry_kg: convert(entry_obj))
        btn_convert.pack(side=LEFT, padx=5, pady=5)


defmain():
    root = Tk()
    root.geometry("300x200+300+200")
    app = Example()
    root.mainloop()

if __name__ == '__main__':
    main()

Solution 3:

Your entry_kg is not known anywhere outside the scope of initUI method. That is why. You could convert it from a method variable to instance attribute to be within reach for class methods by replacing:

entry_kg = Entry(frame_kg)
entry_kg.pack(fill=X, padx=(5, 30), expand=True)

with:

self.entry_kg = Entry(frame_kg)
self.entry_kg.pack(fill=X, padx=(5, 30), expand=True)

Only then:

You can mention it in a class method like:

...
kg = self.entry_kg.get()

That way you if you make your convert a method under Example again:

definitUI(self):
...
defconvert(self):           # is defined under the same scope as initUIprint("clicked")
    kg = self.entry_kg.get() # notice that it now refers to self.e...print(kg)

also don't forget to replace command option as well:

btn_convert=Button(..., command=self.convert)

Or only then:

When outside the class scope, by using dot notation on the object that the class creates:

def main():

    root = Tk()
    root.geometry("300x200+300+200")
    app = Example()
    kg = app.entry_kg.get() # This would return an empty string, but it would still be valid
    root.mainloop()

to be used with global methods, such as the current state of convert you need to make app (the object it is an attribute of) global, or pass it explicitly to the method.

Python Line-by-line Memory Profiler?

2023-09-26T07:58:00.001-07:00

I'm looking to generate, from a large Python codebase, a summary of heap usage or memory allocations over the course of a function's run. I'm familiar with heapy, and it's served m

Solution 1:

I would use sys.settrace at program startup to register a custom tracer function. The custom_trace_function will be called for each line of code. Then you can use that function to store information gathered by heapy or meliae in a file for later processing.

Here is a very simple example which logs the output of hpy.heap() each second to a plain text file:

import sys
import time
import atexit
from guppy importhpy_last_log_time= time.time()
_logfile = open('logfile.txt', 'w')

def heapy_profile(frame, event, arg):
    currtime = time.time()
    if currtime - _last_log_time < 1:
        return_last_log_time=currtimecode= frame.f_codefilename= code.co_filenamelineno= code.co_firstlinenoidset= hpy().heap()
    logfile.write('%s %s:%s\n%s\n\n' % (currtime, filename, lineno, idset))
    logfile.flush()

atexit.register(_logfile.close)
sys.settrace(heapy_profile)

Solution 2:

You might be interested by memory_profiler.

Class Weights For Balancing Data In Tensorflow Object Detection Api

2023-09-26T05:45:00.001-07:00

I'm fine-tuning SSD object detector using TensorFlow object detection API on Open Images Dataset. My training data contains imbalanced classes, e.g. top (5K images) dress (50K ima

Solution 1:

the API expects a weight for each object (bbox) directly in the annotation files. Due to this requirement the solutions to use class weights seem to be:

1) If you have a custom dataset you can modify the annotations of each object (bbox) to include the weight field as 'object/weight'.

2) If you don't want to modify the annotations you can recreate only the tf_records file in order to include the weights of the bboxes.

3) Modify the code of the API (seemed to me quite tricky)

I decided to go for #2, so I put here the code to generate such weighted tf records file for a custom dataset with two classes ("top", "dress") with weights (1.0, 0.1) given a folder of xml annotations as:

import os
import io
import glob
import hashlib
import pandas as pd
import xml.etree.ElementTree as ET
import tensorflow as tf
import random
from PIL import Image
from object_detection.utils import dataset_util

# Define the class names and their weight
class_names = ['top', 'dress', ...]
class_weights = [1.0, 0.1, ...]

defcreate_example(xml_file):

        tree = ET.parse(xml_file)
        root = tree.getroot()
        image_name = root.find('filename').text
        image_path = root.find('path').text
        file_name = image_name.encode('utf8')
        size=root.find('size')
        width = int(size[0].text)
        height = int(size[1].text)
        xmin = []
        ymin = []
        xmax = []
        ymax = []
        classes = []
        classes_text = []
        truncated = []
        poses = []
        difficult_obj = []
        weights = [] # Important linefor member in root.findall('object'):

           xmin.append(float(member[4][0].text) / width)
           ymin.append(float(member[4][1].text) / height)
           xmax.append(float(member[4][2].text) / width)
           ymax.append(float(member[4][3].text) / height)
           difficult_obj.append(0)

           class_name = member[0].text
           class_id = class_names.index(class_name)
           weights.append(class_weights[class_id])

           if class_name == 'top':
               classes_text.append('top'.encode('utf8'))
               classes.append(1)
           elif class_name == 'dress':
               classes_text.append('dress'.encode('utf8'))
               classes.append(2)
           else:
               print('E: class not recognized!')

           truncated.append(0)
           poses.append('Unspecified'.encode('utf8'))

        full_path = image_path 
        with tf.gfile.GFile(full_path, 'rb') as fid:
            encoded_jpg = fid.read()
        encoded_jpg_io = io.BytesIO(encoded_jpg)
        image = Image.open(encoded_jpg_io)
        if image.format != 'JPEG':
           raise ValueError('Image format not JPEG')
        key = hashlib.sha256(encoded_jpg).hexdigest()

        #create TFRecord Example
        example = tf.train.Example(features=tf.train.Features(feature={
            'image/height': dataset_util.int64_feature(height),
            'image/width': dataset_util.int64_feature(width),
            'image/filename': dataset_util.bytes_feature(file_name),
            'image/source_id': dataset_util.bytes_feature(file_name),
            'image/key/sha256': dataset_util.bytes_feature(key.encode('utf8')),
            'image/encoded': dataset_util.bytes_feature(encoded_jpg),
            'image/format': dataset_util.bytes_feature('jpeg'.encode('utf8')),
            'image/object/bbox/xmin': dataset_util.float_list_feature(xmin),
            'image/object/bbox/xmax': dataset_util.float_list_feature(xmax),
            'image/object/bbox/ymin': dataset_util.float_list_feature(ymin),
            'image/object/bbox/ymax': dataset_util.float_list_feature(ymax),
            'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
            'image/object/class/label': dataset_util.int64_list_feature(classes),
            'image/object/difficult': dataset_util.int64_list_feature(difficult_obj),
            'image/object/truncated': dataset_util.int64_list_feature(truncated),
            'image/object/view': dataset_util.bytes_list_feature(poses),
            'image/object/weight': dataset_util.float_list_feature(weights) # Important line
        })) 
        return example  

defmain(_):

    weighted_tf_records_output = 'name_of_records_file.record'# output file
    annotations_path = '/path/to/annotations/folder/*.xml'# input annotations

    writer_train = tf.python_io.TFRecordWriter(weighted_tf_records_output)
    filename_list=tf.train.match_filenames_once(annotations_path)
    init = (tf.global_variables_initializer(), tf.local_variables_initializer())
    sess=tf.Session()
    sess.run(init)
    list = sess.run(filename_list)
    random.shuffle(list)  

    for xml_file inlist:
      print('-> Processing {}'.format(xml_file))
      example = create_example(xml_file)
      writer_train.write(example.SerializeToString())

    writer_train.close()
    print('-> Successfully converted dataset to TFRecord.')


if __name__ == '__main__':
    tf.app.run()

If you have other kinds of annotations the code will be very similar but this one unfortunately will not work.

Solution 2:

The Object Detection API losses are defined in: https://github.com/tensorflow/models/blob/master/research/object_detection/core/losses.py

In particular, the following loss classes have been implemented:

Classification losses:

WeightedSigmoidClassificationLoss
SigmoidFocalClassificationLoss
WeightedSoftmaxClassificationLoss
WeightedSoftmaxClassificationAgainstLogitsLoss
BootstrappedSigmoidClassificationLoss

Localization losses:

WeightedL2LocalizationLoss
WeightedSmoothL1LocalizationLoss
WeightedIOULocalizationLoss

The weight parameters are used to balance anchors (prior boxes) and are of size [batch_size, num_anchors] in addition to hard negative mining. Alternatively, the focal loss down weighs well classified examples and focusses on the hard examples.

The primary class imbalance is due to many more negative examples (bounding boxes without objects of interest) in comparison to very few positive examples (bounding boxes with object classes). That seems to be the reason why class imbalance within positive examples (i.e. unequal distribution of positive class labels) is not implemented as part of object detection losses.

Python - Updating Json Objects In A File

2023-09-26T04:39:00.001-07:00

I have a (valid) JSON file contaning JSON objects, each object taking up one file line. These JSON objects map to Python dicts, and are in ascending order by numerical IDs. I am w

Solution 1:

You cannot do what you want, because you can only surgically replace fixed length records in a file. You cannot replace the line with something that is longer or shorter, as files don't store data as separate lines.

In addition, your code doesn't take into account that on some platforms (like on Windows) lines in text files are separated by two characters, representable in Python as \r\n, but these line endings are translated for you to the single character \n. Your line length calculation doesn't take that into account.

You instead have to rewrite everything following that line. You'd be better of using a database; sqlite3 comes bundled with Python, take a look at that.

Solution 2:

defupdate_JSON_file( self, file_path, obj_IDs, props ):
    decoder = json.JSONDecoder()
    with io.open( file_path, 'r+', encoding='utf=8' ) as files:
        IDs_iter = iter( obj_IDs )
        fl = files.read().splitlines(True)
        ID = IDs_iter.next()
        for i,line inenumerate(fl):
            # ID = IDs_iter.next()# I think the 'ID' should be initialized out of the loop# and iterated only if the condition is trueifstr(ID) in line:
                JSON_obj = decoder.raw_decode(line)[0]
                for key in props.keys():
                    JSON_obj[key] = props[key] 
                JSON_obj = json.dumps( JSON_obj, ensure_ascii=False )
                if'},\n'in line: JSON_obj += ',\n'
                JSON_obj = unicode( JSON_obj )
                fl[i] = JSON_obj
                try:
                    ID = IDs_iter.next()
                except StopIteration:
                    break
        files.seek(0)
        files.write(''.join(fl))
        returnTrue

Comparing Two Lists And Only Printing The Differences? (xoring Two Lists)

2023-09-26T02:26:00.001-07:00

I'm trying to create a function that takes in 2 lists and returns the list that only has the differences of the two lists. Example: a = [1,2,5,7,9] b = [1,2,4,8,9] The result sho

Solution 1:

You basically want to add an element to your new list if it is present in one and not present in another. Here is a compact loop which can do it. For each element in the two lists (concatenate them with list1+list2), we add element if it is not present in one of them:

[a for a in list1+list2 if (a not in list1) or (a not in list2)]

You can easily transform it into a more unPythonic code with explicit looping through elements as you have now, but honestly I don't see a point (not that it matters):

defxor(list1, list2):
    outputlist = []
    list3 = list1 + list2
    for i inrange(0, len(list3)):
        if ((list3[i] notin list1) or (list3[i] notin list2)) and (list3[i] notin outputlist):
             outputlist[len(outputlist):] = [list3[i]]
    return outputlist

Solution 2:

Use set is better

>>>a = [1,2,5,7,9]>>>b = [1,2,4,8,9]>>>set(a).symmetric_difference(b)
{4, 5, 7, 8}

Thanks to @DSM, a better sentence is:

>>>set(a)^set(b)

These two statements are the same. But the latter is clearer.

Update: sorry, I did not see the last requirement: cannot use set. As far as I see, the solution provided by @sashkello is the best.

Solution 3:

Note: This is really unpythonic and should only be used as a homework answer :)

After you have sorted both lists, you can find duplicates by doing the following:

1) Place iterators at the start of A and B

2) If Aitr is greater than Bitr, advance Bitr after placing Bitr's value in the return list

3) Else if Bitr is greater than Aitr, advance Aiter after placing Aitr's value in the return list

4) Else you have found a duplicate, advance Aitr and Bitr

Solution 4:

This code works assuming you've got sorted lists. It works in linear time, rather than quadratic like many of the other solutions given.

defdiff(sl0, sl1):
    i0, i1 = 0, 0while i0 < len(sl0) and i1 < len(sl1):
        if sl0[i0] == sl1[i1]:
            i0 += 1
            i1 += 1elif sl0[i0] < sl1[i1]:
            yield sl0[i0]
            i0 += 1else:
            yield sl1[i1]
            i1 += 1for i in xrange(i0, len(sl0)):
        yield sl0[i]
    for i in xrange(i1, len(sl1)):
        yield sl1[i]

printlist(diff([1,2,5,7,9], [1,2,4,8,9]))

Solution 5:

Try this,

    a = [1,2,5,7,9]
    b = [1,2,4,8,9]
 printset(a).symmetric_difference(set(b))

Spacy Custom Sentence Spliting

2023-09-25T23:23:00.001-07:00

I using Spacy for custom sentence spliting and i need to parametrized the custom_delimeter/word for sentence spiting but i didnt find how to pass as an arugument here is the functi

Solution 1:

You could turn your component into a class that can be initialized with a list of delimiters? For example:

classMyCustomBoundary(object):
    def__init__(self, delimiters):
        self.delimiters = delimiters

    def__call__(self, doc):  # this is applied when you call it on a Docfor token in doc[:-1]:
            if token.text in self.delimiters:
                doc[token.i+1].is_sent_start = Truereturn doc

You can then add it to your pipeline like this:

mycustom_boundary = MyCustomBoundary(delimiters=['...', '---'])
nlp.add_pipe(mycustom_boundary, before='parser')

Python Chained Get() Method With List Element Inside Json

2023-09-25T22:23:00.001-07:00

[Python 2.7] I have a JSON source that doesn't always return the full list of expected keys. I'm using chained gets() to address this. d = {'a': {'b': 1}} print(d.get('a', {}).get

Solution 1:

I agree with @stovfl that writing your own lookup function is the way to go. Although, I don't think a recursive implementation is necessary. The following should work well enough:

def nested_lookup(obj, keys, default='NA'):
    current = obj
    for key in keys:
        current = current if isinstance(current, list) else [current]
        try:
            current = next(sub[key] forsubincurrentifkeyinsub)
        exceptStopIteration:
            returndefaultreturncurrentd = {'a': {'b': [{'c': 2}, {'d': 3}]}}

print nested_lookup(d, ('a', 'b', 'c'))  # 2print nested_lookup(d, ('a', 'b', 'd'))  # 3print nested_lookup(d, ('a', 'c'))       # NA

The class approach doesn't seem great because you're going to be creating a lot of unnecessary objects and if you're ever trying to lookup a node that isn't a leaf, then you're going to wind up with a custom object rather than the actual node object.

Solution 2:

Question: I can't use a get() ... because lists don't support the get().

You can implement your own list.get(), for instance:

classmyGET(object):
    def__init__(self, data):
        ifisinstance(data, dict):
            self.__dict__.update(data)
        ifisinstance(data, list):
            for d in data:
                self.__dict__.update(d)

    defget(self, key, default=None):
        ifhasattr(self, key):
            _attr = object.__getattribute__(self, key)
            ifisinstance(_attr, (list, dict)):
                return myGET(_attr)
            else:
                return _attr
        else:
            return default

d = {'a': {'b': [{'c': 2}]}}
myGET(d).get('a', {}).get('b', []).get('c', 'NA')
>>> 2

myGET(d).get('a', {}).get('b', []).get('d', 'NA')
>>> NA

A recursive Solution, no need to chaining, for instance:

defget(_dict, subkey, default):
    def_get(_dict, key, deep):
        if key == subkey[deep]:
            if deep == len(subkey) - 1:
                return _dictelse:
                return _get(_dict, None, deep + 1)

        elifisinstance(_dict, dict):
            for k in _dict:
                match = _get(_dict[k], k, deep)
                if match: return match

        elifisinstance(_dict, list):
            for e in _dict:
                match = _get(e, None, deep)
                if match: return match

    ifnotisinstance(subkey, (tuple, list)):
        subkey = (subkey)

    _r = _get(_dict, None, 0)
    ifnot _r: return default
    else:      return _r

get(d, 'c', 'NA')
>>> 2

get(d, 'd', 'NA')
>>> NA

# get a inside b
d = {'a': {'b': [{'a': 3}, {'c': 2}]}}
get(d, ('b', 'a'), 'NA')
>>> 3

Tested with Python: 3.4.2 and 2.7.9

Private Unicode Character Displays Differently In Python 3 Interpreter

2023-09-25T21:19:00.001-07:00

So I created a unicode character privately using Private Character Editor on Windows 10. The character was saved with the code E000. I copied it from the Character Map and pasted i

Solution 1:

Python isn't really the interesting part of this, rather the shell or terminal is. In our case, Windows uses special code points to represent private character encodings. To get those, you need to get a hex dump of the character on a shell in Windows, then you can render the character in Python.

NOTE: Use Unicode points E021 or higher, since lower number code points are usually used for control, and it seems that the Windows shell that the python interpreter and IDLE use doesn't let you override those with private characters.

Demonstration

I tested your issue by generating a private character of my own. I will put an image of my test here since it wouldn't be rendered properly in text here on Stack Overflow.

Explanation

I used the Character Map program in Windows 10 to copy the symbol and paste it into my python environment. The environment may truncate it on the right since it is a wide character and the environment didn't seem to like that. (I moved the cursor around to get it to render full-width.)

Then I proceeded to get the hexdump of the code point by encoding the character using the default utf-8 encoding, which turned out to be \xee\x80\xa1 as a bytes object.

Next I printed the data as a string to show you a common error, and what would be printed if you attempted to print a string of those bytes.

Then, I printed b'\xee\x80\xa1', which is how you would actually use the symbol in your software.

Solution 2:

You can use the \u escape sequence in your Python source code, like so:

my_unicode_string = 'This is my Unicode character: \ue000'print(my_unicode_string)

Looking For A More Pythonic Way To Access The Database

2023-09-25T19:17:00.001-07:00

I have a bunch of python methods that follow this pattern: def delete_session(guid): conn = get_conn() cur = conn.cursor() cur.execute('delete from sessions where guid

Solution 1:

You could write a context manager and use the with statement. For example, see this blog post:

http://jessenoller.com/2009/02/03/get-with-the-program-as-contextmanager-completely-different/

Also the python documentation has a sample that pretty much matches your needs. See section 8.1 on this page, in particular the snippet that begins:

db_connection = DatabaseConnection()
with db_connection as cursor:
    cursor.execute('insert into ...')
    cursor.execute('delete from ...')
    # ... more operations ...

https://docs.python.org/2.5/whatsnew/pep-343.html

Solution 2:

Careful about that execute, the second argument needs to be [guid] (a list with just one item). As for your question, I normally just use a class encapsulating connection and cursor, but it looks like you may prefer to use an execution context object whose __enter__ method gives you a cursor while __leave__ commits or rollbacks depending on whether the termination was normal or by exception; this would make your code

def delete_session():
    withget_cursor() as cur:
        cur.execute(etc etc)

If you like this style, let us know and I'll show you how to write get_cursor. Others will no doubt propose a decorator instead, so you'd write:

@withcursordefdelete_session(cur):
    cur.execute(etc etc)

but I think this makes commit/rollback, among other issues, a bit murkier. Still, if this is your preference, again let us know and I can show you how to write that form, too.

Solution 3:

"I have a bunch of python methods that follow this pattern:"

This is confusing.

Either you have a bunch of functions, or you have a bunch of methods of a class.

Bunch of Functions.

Do this instead.

classSQLFunction( object ):
    def__init__( self, connection ):
        self.connection = connection
    def__call__( self, args=None):
        self.cursor= self.connection.cursor()
        self.run( args )
        self.cursor.commit()
        self.cursor.close()

classDeleteSession( SQLFunction ):
    defrun( self, args ):
        self.cursor.execute( "statement" )

delete_session = DeleteSession( connection )

Your function declarations are two lines longer, but essentially the same. You can do func1( args ) because it's a callable object. The rest of your program should remain unchanged.

Bunch of Methods in One Class.

classSomeClass( object ):
    def__init__( self, connection ):
        self.connection= connection
    defsql_execute( self, statement, args= None)
        self.cursor= self.connection.cursor() 
        self.cursor.execute( statement, args if args isnotNoneelse [] )
        self.connection.commit()
        self.cursor.close()
    defdelete_session( self ):
        self.sql_execute( "statement" )

All your methods can look like delete_session and make use of a common sql_execute method.

Solution 4:

A decorator?

classSqlExec:def__init__(self, f):
      self.f = f
   def__call__(self, *args):
      conn = get_conn() 
      cur = conn.cursor()
      cur.execute(self.f (*args))
      conn.commit()
      conn.close()

@SqlExecdefdelete_session(guid):
      return"delete from sessions where guid=%s" % guid

Solution 5:

According to the docs, if you were using SQLite3, you wouldn't even need a Cursor which, as the docs say, is "often superfluous".

Instead you can use the shortcut methods executeexecutemany and executescript directly on the connection object:

import sqlite3

persons = [
    ("Hugo", "Boss"),
    ("Calvin", "Klein")
    ]

con = sqlite3.connect(":memory:")

# Create the table
con.execute("create table person(firstname, lastname)")

# Fill the table
con.executemany("insert into person(firstname, lastname) values (?, ?)", persons)

# Print the table contents
for row in con.execute("select firstname, lastname from person"):
    print row

print"I just deleted", con.execute("delete from person").rowcount, "rows"

Tesseractnotfound - Pytesser

2023-09-25T15:34:00.001-07:00

I'm trying to do OCR using pytesser downloaded from HERE. Here is the code of pytesser.py try: import cv2.cv as cv OPENCV_AVAILABLE = True except ImportError: OPENCV_AV

Solution 1:

Following changes in pytesser.py solved my problem, there is no problem with the paths that are set.

Changes are as follows:

PROG_NAME=tesseract changed to PROG_NAME=tesseract.exe

In the function image_to_string() added f.close() after txt=f.read()

That't it :)

Axes3d.plot_wireframe(x,y,z) Error

2023-09-25T14:47:00.001-07:00

I'm trying to learn Python through a tutorial on youtube and I'm having some difficulies working with 3D graphs. Long stories short, I continuously get (if Z.ndim != 2: AttributeE

Solution 1:

I walked around this problem by doing two things.

import numpy as np
making the z-axis a multidimensional array

#My 3d graph

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
import numpy as np

figure = plt.figure()
axis = figure.add_subplot(111, projection = '3d')

x = [1,2,3,4,5,6,7,8,9,10]
y = [5,6,7,8,2,5,6,3,7,2]
z = np.array([[1,2,6,3,2,7,3,3,7,2],[1,2,6,3,2,7,3,3,7,2]])

axis.plot_wireframe(x, y, z)

axis.set_xlabel('x-axis')
axis.set_ylabel('y-axis')
axis.set_zlabel('z-axis')

plt.show()

Take special note of the z variable. If z is not multidimensional, it will throw an error.

Hope it solves your problem

Solution 2:

Running your code with either Python 2.7.10 or Python 3.6.0, with matplotlib version 2.0.2, yields the same image with no error:

This is not a wireframe though, and a simple ax.plot(X, Y, Z) would have generated it. As DavidG and ImportanceOfBeingErnest cleverly mentioned, it makes no sense to pass 1D lists to the wireframe function, as X, Y and Z should be two-dimensional.

The following code (an example taken from the matplotlib official documentation) shows exactly how the parameters of the plot_wireframe function should be (using numpy arrays):

from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
import numpy as np

'''
def get_test_data(delta=0.05):

    from matplotlib.mlab import  bivariate_normal
    x = y = np.arange(-3.0, 3.0, delta)
    X, Y = np.meshgrid(x, y)

    Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
    Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
    Z = Z2 - Z1

    X = X * 10
    Y = Y * 10
    Z = Z * 500
    return X, Y, Z

'''


fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x, y, z = axes3d.get_test_data(0.05)

ax.plot_wireframe(x,y,z, rstride=2, cstride=2)

plt.show()

The output image is a true wireframe:

Printing x.shape, for instance, yields you (120, 120), showing that the array is two-dimensional and have 120 positions in the first dimension and 120 positions in the second one.

Solution 3:

I had the exact problem (example from video not working though exactly copied). Without looking into the source code I'm assuming a reality check was added to matplotlib 2.1.0 that NOW stops 1D arrays from being used in plot_wireframe. Changing that method call to simply "plot" did indeed fix the problem.

Solution 4:

The command ax.plot_wireframe(x,y,z, rstride=2, cstride=2) is creating the problems with the latest versions. Try using:

ax.plot(x,y,z)

This will definitely solve your issues. Python has been known for being inconsistent with the older libraries. I am getting this image as the output: This is the 3d Image I am getting