[This may be my longest-sitting draft that I finally made into a post. I started it in June 2017, shortly after this graphic made the rounds.]

After studying phonetics in my first year at Tel-Aviv University, I developed a pet theory. See, every phenomenon we came across that was "unique" (or rare, or marked) seemed to have somehow skipped over the Hebrew language. No crazy nasals, no retroflexes, no gutturals (in the standard Israeli dialect), no clicks, no ingressives, no voiced alveolar lateral fricative, just the five canonical vowels, et cetera. My thoughts were, since Modern Hebrew is a revived language co-learned by people from very distinct linguistic backgrounds in a relatively messy (high-entropy) distribution, the phonetic portion of it evolved to a low-common-denominator, most phonetically boring language out there.

But where there's data, there's a chance to test out pet theories. So as soon as I got word of the phoible dataset I immediately jumped and put my theory to the test (then waited 15 months for absolutely no reason to actually publish my findings).

Phoible is an open, simple-format data source for phonetic inventories of language. A few clicks, and you have a table of all phonemes across all languages. Thus my definition for boringness of language converged to:

A language is boring if it chooses boring sounds for its inventory.

Now I need to figure out which sounds (phonemes) are boring, but that I just define based on their frequency across languages, which gives me a clear algorithm:

1. Calculate each phoneme's frequency in the database
2. For each language, calculate average phoneme frequency from its inventory
3. Rank languages from high to low average (high = boring)

That's it! For my first finding, Modern Hebrew got a score of 0.415, which ranked it 1,426th most boring in a field of 2,155 languages, an utter refutation of my hypothesis. The most boring language according to this metric is Southern Nuautl with a score of 0.764; the most interesting is !xóõ (yes, that's a click sound it's starting with) with an astounding 0.103. The average score was 0.464 and the score progression is a normal-distributed thing of this:

(CDF = cumulative distribution function, meaning: y many languages have score up to x)

Here's a taste of some languages I thought could be of interest. Check out that lovely long tail of Igbo, which has may phonemes but a lot of the frequent ones; or how fast Quechua plummets from frequent to semi-frequent to rare phonemes; or how boring Swahili, a high-contact language dominating the 20's of the x-axis, is (it's what I expected the Hebrew situation to be, and even so its score is a very unboring 0.292); or how English falls so quickly in the beginning, with all its weird vowels and labiodentals and taps and flaps.

This could mean a few things about why I didn't get what I expected, as well as some other unrelated reasons:

1. Like always, data is dirty (or at least, this data, for my analysis needs). In this case, more well-documented languages may have more phonemes in their dataset, probably leading to some that are rare, than languages with less extensive research done upon.
2. My metric must suck. See how Hebrew has the fewest phonemes in the selected sample? That's gotta account for boringness and yet, with my mean it doesn't. Look how many frequent phonemes Swahili has, and yet its average is very low. Let's consider some other metrics ("Future work". Remind me to upload the data if I don't do so soon):
   1. Number of phonemes (a boring metric for a boringness question)
   2. % of phonemes above a boringness threshold p.
   3. Area under the boringness curve (this is just the sum of boringnesses again – but maybe cut it off at some point?)
   4. Deep neural net trained on all these features with the single data point <Hebrew, TRUE>.
3. A bug in my code. As soon as I find it I see if I can look deeper. I mean it's kind of a miracle that I have the post-processed TSVs around, to be honest.

אני הולך היום לראות הופעה של נורית גלרון. כיף לי!

בעודי מכין את עצמי אתמול בשמיעת פלייליסט מכובד של שיריה בעבודה, הרהרתי לעצמי: רגע רגע רגע. מה פתאום גשם מכסה את אספלט הכביש ביום הכי קר בשנה? הרי בדרך כלל כשיורד גשם יש עננים, וימים מעוננים נוטים להיות לא הכי קרים כי הם מהווים שכבת בידוד או משהו. בכל מקרה למיטב נסיוני, אחרי שהגשם מפסיק והעננות מתפזרת נהיה דווקא יותר קר, לא יותר חם. לא?

מה יקרה אם פתאום נבדוק את ההשערה? נתחיל בחיפוש אחר דאטאסט זמין וקל לניתוח. איפה אפשר לחפש היום דבר כזה? בכל מקום, אבל אני ממליץ למי שעוד אין לפתוח משתמש בקאגל. חיפוש קטנטן של in:datasets weather מעלה מיידית עשרות אפשרויות. אז מזג אוויר ישראלי לא מצאתי, אבל התוצאה הראשונה שכן קרצה לי היתה זו של אוסטין, טקסס, עיר עם טמפרטורות יחסית דומות למישור החוף וכמות משקעים דומה בחודשי החורף.

אמנם קצת מבאס שהנתונים הם רק לשלוש שנים שלמות (2014-2016) עם קצת שאריות לפני ואחרי, אבל הכל מתקזז אל מול המופלאות של קאגל. לא צריך להוריד קובץ, לא צריך להרים סביבה, כלום. פשוט פותחים קרנל (סביבת כתיבה והרצה של קוד בתוך "מחברת" מתעדכנת המורכבת מתאים; קאגל תומכים בפייתון ואר [R]) על האתר והוא מריץ לעצמו את הכל ברקע. אז התכנון היה להשתפשף עם הקוד על השלוש שנים של אוסטין ואז ללכת לגוש מידע גדול שאשיג ממקור אחר, אבל כבר המיני-ניסוי הזה הבהיר שההיפותזה שלי לא מחזיקה מים (הא!) ואני מציג בפניכם את תוצאות הכשל הגדול:

מחברת הפייתון המכשילה.

הנתון הסותר: ב-10 בינואר, 2015, נמדדו באוסטין 36 מעלות פרנהייט (בערך 2 בלשון אדם), הכי מעט באותה שנה, וירדו 0.3 אינטשים של משקעים, שהם כ-7.5 מ"מ, בהחלט מספיק כדי לכסות אספלט של כביש.

הישארו עמי לניסוי הבא, בו לפני הופעה של יזהר אשדות אחטן כבד בבלנדר ואבדוק אם אפשר לשתות אותו.

[עדכון, 31/1: בעצתו של ליאון התותח הלכתי לאתר השירות המטאורולוגי, שם ניתן להוריד נתוני מזג אוויר משלל תחנות בארץ. הורדתי קובץ עם כל ימי הגשם כפי שנמדדו בבית דגן בין השנים 1983 ל-1992, העשור לפני שהשיר יצא (כולל). הורדתי קובץ נוסף עם הטמפרטורות היומיות. הרמתי מחברת במחשב האישי. היו שמונה ימי גשם כאלה בעשור הנ"ל. הנה המחברת, והנה צילומסך של הסוף שלה, כולל התוצאה בתא האחרון.

]

Here's the path I took, as a non-US-citizen, before moving to the US and starting a CS PhD program. I took the GRE and TOEFL around summer of 2015 and moved in August 2016.

I provide it as a reference for similar-minded folk, but keep in mind times change, circumstances vary, and I may have forgotten crucial steps. In any case, enjoy.

(Created using GraphViz)

This February, I gave an interview to Ido Kenan on Galei Tzahal (IDF Radio) about Google's upgraded Machine Translation system, including its claims that it learns an intermediary abstract language representation, an "Interlingua".

You can listen to the interview here, or read my writeup here on Kenan's blog. Problem is, it's all in Hebrew! Well, what better than to use the fancy new Google Translate to render the thing into English?

Here it is, untouched. See how much you understand. (Retrieved March 16, 2017)

Abstract language behind Google Translation

Following a report by Google , technology sites reported excitedly AI behind Google Trnsliit invented language. Reality only a little less Krief: AI discovered a more efficient way to translate between languages, using interlingua – abstract language that links between the two languages.

Yuval Pinter, a doctoral student in computer science language processing at Georgia Tech, explained the significance of the innovation program. He told us in an interview ahead: "Google Translate recently deployed a new translation engine that replaced the old, gradually starting in September. The system implements research ideas out there for decades and has only recently become feasible. In November the team published a paper in which he explained how the system actually succeeds in presenting a trained human language in the abstract, that made possible to translate directly between languages that the system had never met them examples. The performance showed not perfect, but certainly there is a conceptual leap forward.

"For most of the history of machine translation, the prevailing attitude was' based-phrases. According to this view, Big arthritis source sentence into parts (phrases) that seem reasonable to translate, translate them separately and then build the sentence in the target language as much as possible try to score its syntax and logic. The knowledge of each of these stages can be built automatically: show in a lot of examples of translated sentences and grammatical sentences from one language, and it 'learns' how to break a sentence, how to translate each phrase, and how to catch up. But still there are a lot of human intervention at every stage and the aisles between them. For example, if we take the phrase 'the Prime Minister yesterday visited the power station, and ask for translation, the system will need to know, among other things: Primary government, one that translates this phrase The Prime Minister and head the Government; S'bikr, this physical sense (visited) and rhetorical (criticized); Verb translated phrase should follow the topic; That yesterday will have a comma after it; Q-Jeb, not in it at; And more. The first two rules will be studied at a reasonable level automatically, but the last three probably require human hand encode specific knowledge about Hebrew and English. These systems were common until today.

"In the new system there is a massive application of technology, which until a few years ago was largely theoretical amusement, and was made possible thanks to advances in computing power and configuration process, and aggregation of data volumes magnitudes above what was acceptable. The new algorithm, there are many rules that people have written, or at least directed on the basis of knowledge of any language, computer builds his own rules. The main difference is obtained directly translating a whole sentence complete sentence, and therefore do not need to know in advance the language as long as there is enough data.

, The last article that Google released showed not only the translation process is the same between pairs of languages, but Sctotzr effects of learning, the system builds a kind of general representation, not language-dependent, the court. We said that today a translation between two languages hung in the mere existence of millions of sentences that we know their translation. Make it easier for couples languages like English-French, for example, when the Canadian Parliament's protocol or mechanism in the EU goes multilingual uniform. But what happens when you try to build a system of translation from Korean to Swahili, Hebrew or Spanish? What they did today this translation through an intermediate language (in practice, always English), from Korean to English and Swahili. It is also much more logical application – let's say there are 100 languages, so have about 200 interfaces, where about 10,000 if you want a direct translation from any language to any language, it is not applicable. One drawback of this approach is the effect of "broken telephone" as the translations are not perfect, but there are also a matter of lost data. For example, Hebrew and Spanish – the phrase 'the cat eats the cheese' will be translated into English through the cat is eating the cheese, and losing here that the cat is female, a distinction that both Hebrew and Spanish, but no English. 'Intermediate language, abstract Google found her testimony, however, is rich enough to contain the information of grammatical gender, and that preservation GATA transition to Spanish.

"" Intermediate language "itself is, as mentioned, abstract. She lives in mathematically impossible to pronounce the words and phrases in it. Google showed how it exists? Took sentences translated from Japanese to English and Korean to English, and only a trained them. Then he said to translate sentences from Japanese to Korean, things she had not seen during training. Translations were reasonable and beautiful competed last through the English translation. Then a little 'Help' system with a relatively small amount of samples translation from Japanese to Korean, then the translations were as good as the direct model, trained many examples of Japanese-Korean.

"According to the notification of Google, the system has been deployed, this means that the system is now active as we are using Google Trnsliit. English translations for all languages and vice versa should be better than before. It seems we have not deployed the common model, but if it happens, also from Hebrew into other languages quality improves. The Hebrew they did not publish the results, I do not know if you speak Hebrew enjoying significant improvement. "

I have yet to see a piece trying to tie the two together (please correct me if I'm wrong).

My claim is simple: citing the nationwide 56.5% figure as a strong indicator for voter apathy is somewhat misleading. If a Californian feels they don't see the point in voting (and registering beforehand), it's different than a Pennsylvania voter (in this elections cycle at least, but pretty much usually). It's unfair, but understandable, if there's a (say) 15 point difference in their turnout rates.

Let's look at the numbers then, shall we? On the x axis, we'll place the ultimate victory margin (collected Thursday from Wikipedia) as a proxy for how inclined an average voter was to believe that his vote would be crucial. It's not a perfect proxy of course, as there were some state-level surprises. Maybe poll margins prior to registration deadlines would have been a better one. The y axis will denote the voting turnout (collected from electproject.org).

Before the chart, observational data: of the 11 states with highest voter turnout rate, 10 ended up with a margin under 5%. Of the 10 states with margin under 4%, only one had a turnout of less than 60%. Now you can look at the chart, including a simple linear trend line.

Forgive my dataviz-unsavviness. I wish I knew how to add state labels to each point on gsheets.

As you can see, the results are pretty straightforward. With a not-bad correlation of 0.21, it seems voters chose to turn out based on how close they anticipated the race to be in their state. I didn't leave out the outliers but they're not shown in this chart (DC is always ridiculous, this time with an 86% victory margin. Hawaii significantly undervoted with a 34% turnout, way under the next, California at 45.5%). It was cool to see Utah as a special case here with its 3-way race – a 19% D-R margin brought significantly less people to the polls than Montana or Washington state who ended up with about the same margin.

All in all, the voters who mattered in this Presidential election (tough phrasing but that's the way it is) came in at about 65%, much higher than the national average.

It's worth noting that the numbers, even for the swing states, are still low compared to most of the democratic world. But I'll also note that the US has other factors going for it, such as the no-day-off thing, or the huge amount of expats allowed to vote, which is a unique characteristic. According to electproject, these compose roughly 2% of the eligible electorate. ]]> https://blazinghyphens.wordpress.com/2016/11/10/turnout-burnout/feed/ 0 1190 blazinghyphens 2016-state-voting https://blazinghyphens.wordpress.com/2016/06/26/georgia-tech-at-naacl-2016/ https://blazinghyphens.wordpress.com/2016/06/26/georgia-tech-at-naacl-2016/#respond Sun, 26 Jun 2016 10:31:31 +0000 http://blazinghyphens.wordpress.com/2016/06/26/georgia-tech-at-naacl-2016/ (Original author: Jacob Eisenstein)

אין מצב להשאיר את הספינה בלי רועה.

אגב, למה שפורר קיבלה שם קריטורת אילוסטרציה ורוט הוותיקה בהרבה לא?

(אין צורך לענות )

אתר פסטיבל הסרטים ה-ל"א עלה לאוויר בשעה טובה. אפילו שמו תוכניה ברמה היומית של הסרטים עם חלוקת שורות לאולמות.

את ציר הזמן, לעומת זאת, פקדה פשרה מוזרה (שלא לומר מבלבלת שועלים): הימים מסודרים מימין לשמאל. השעות בתוך כל יום מסודרות משמאל לימין. ובתוך כל משבצת זמן, הס פן תעיר, נכתב שם הסרט בכתב אשורי מימין לשמאל. מבולבלים? חכו שתראו את צילום המסך.

פינת סטירה

*בדגש דווקא כתבתי פה ושם: הנה והנה והנה והנה והנה והנה והנה והנה והנה והנה והנה והנה והנה והנה והנה והנה. והצגתי ביסקול וארגננו בהמעבדה טרעק צ'אלאנג'. ועוד ועוד.

** אגב, יש לי גם טאמבלְר. אותו שם משתמש.