SHRIPHANI PALAKODETY: SHRIPHANI PALAKODETY

Indic Language Identification

Thu, 19 Nov 2020 11:00:00 UT

We discuss language identification of noisy, romanized text - an un-addressed but critical problem in Indic text mining, and release a language-identification utility. We then measure geographic extents of language use in India. Summary of a WNUT 2020 paper.

What aspects of Indic (possibly Arabic, Malay, etc.) text mining make it a more challenging undertaking than Western text mining. Our experience suggests:

Noisy text
Romanization
Multilinguality

Noisy text refers to all forms of spelling and grammar disfluencies. English is a (distant) second language to most Indians and simple word-frequency based methods can fail.

Romanization is a widespread phenomenon in informal settings like social media and advertising. Hindi and other vernaculars are mostly written in the Latin alphabet. Our experience suggests this is the majority of the content authored in Indian social media.

Multilinguality is a consequence of a high level of linguistic diversity in the Indian subcontinent. Any substantial corpus acquired contains a significant English, Romanized Hindi, Devanagari Hindi and several vernaculars (Romanized or native).

Language Identification is thus a critical first step in any Indic text-mining task. NLP pipelines and annotators are usually monolingual.

Existing resources for building good language identification systems focus on clean, well-formed text authored in the native script. Highly noisy, Romanized text is un-addressed by all existing tools.

We present a solution precisely for this problem - a lang-id system that works well on noisy social-media text, and handles Romanized Indian text content.

Our solution is extremely annotation-efficient - a mere 260 annotations are sufficient to construct a large data set of 2.4 million labeled documents.

Our solution uses a beautiful result on polyglot training from earlier this year.

Training a single skipgram model on a large multilingual corpus yields clean, well-separated monolingual clusters.

Our pipeline acquires these monolingual components, a skilled annotator labels a few (10 or so) documents from each cluster and we just expand these labels to the rest of the cluster members (we call these weak labels).

The end-result is a large labeled data set representative of social media content (the type that data-mining professionals are most interested in).

The resulting fasttext model is available in the links below.

Geographic Extents

With our new language-identification system, and weak geo-labels from our corpus, we can estimate the geographic extent of languages. We observe that the estimate aligns with field-surveys like the Linguistic Survey of India. Hindi seems to be confined to an area popularly known as the Hindi belt, Romanization is the dominant form of expression in most of India. Online English usage as a percentage of the conversation is higher in Southern India. Bilinguals from areas adjacent to the Hindi belt use Hindi and their native language. As we enter Southern India, Hindi is nearly non-existent on social media.

Bi / Multi Linguality

We next estimate what fraction of users post in multiple languages and what pairings (combinations) are most prevalent. The bulk of language-pairs we observe in bi-lingual users contain English, and bilinguals using 2 South Indian languages are extremely rare.

Richmond Carvers

Tue, 02 Jun 2020 18:15:54 UT

Third place in the Richmond carvers show for this maple panel.

Maple Panel

Mon, 11 May 2020 21:14:53 UT

A new piece carved in maple during this lockdown period.

Kitchen Tools

Mon, 30 Mar 2020 06:49:34 UT

From the COVID–19 shutdown, 2 projects - a coffee scoop in cherry and a spatula in jatoba.

Polyglot Word Embeddings Discover Language Clusters

Mon, 03 Feb 2020 19:45:16 UT

Polyglot word embeddings obtained by training a skipgram model on a multi-lingual corpus discover extremely high-quality language clusters.

These can be trivially retrieved using an algorithm like $k-$Means giving us a fully unsupervised language identification system.

Experiments show that these clusters are on-par with results produced by popular open source (FastText LangID) and commercial models (Google Cloud Translation).

We have successfully used this technique in many situations involving several low-resource languages that are poorly supported by popular open source models.

This blog post covers methods, intuition, and links to an implementation based on 100-dimensional FastText embeddings.

Background

The skipgram model takes a word as input and predicts its context.

The pipeline maps a one-hot representation of a word in the vocabulary $\mathcal{V}$ to a dense, real-valued vector called a word embedding.

The training scheme for this model involves words and the contexts they appear in (the positives), and negative examples obtained using negative sampling.

We’ll henceforth concern ourselves with 100-dimensional FastText word embeddings and refer to them as FastText-100. These results are consistent with the 300-dimensional variant as well.

A document embedding is a single real-valued vector for a full document (sentence, paragraph, social media post etc.).

There are several popular ways of obtaining these but we’ll just use the FastText default:

obtain the word-vectors for all words in the document,
normalize them, and
average them to obtain a single 100-dimensional vector for the whole document.

Polyglot Embeddings

I am going to use a pre-processed Europarl corpus sourced from here.

After a preprocessing step:

The text is lowercased and punctuation stripped.
Each line corresponds to a distinct document, and
The full corpus is shuffled.

The corpus contains 21 languages. The documents are shuffled and (polyglot) FastText-100 embeddings are trained on this 21-language corpus.

You can download it (and models and everything) here

We can visualize the document embedding space using a TSNE plot:

The document embedding space and 21 clusters discovered by $k$-Means.

We observe 21 clear, well defined clusters.

Manual inspection shows that there is exactly one cluster per language in the corpus.

A simple clustering algorithm like $k-$Means is able to retrieve these clusters and on inspection we notice that the corpus is split into 21 parts - each containing documents written in exactly one language.

A Language Identification System

There are two undiscussed steps - picking a value of $k$, and assigning a language to one of these language clusters.

The latter is easy to solve - human annotators assign the dominant language in each cluster as that cluster’s language. A subsequent cluster membership test can be used for a test document.

What remains to be discussed is picking a good $k$ value for the $k$-Means algorithm.

In our experience, popular methods like the silhouette heuristic, or the elbow method are quite capable of producing a reasonable value for $k$.

For the Europarl corpus above, here is a silhouette plot for $k=21$:

And here is the elbow plot:

In both cases, we see 21 as the optimal number of clusters for this corpus.

How Good Are These Clusters?

We run human evaluations on several corpora and report performance against popular open and commercial models. These corpora contain several low-resource languages and thus popular language identification systems are unable to identify them at times:

$\mathcal{D}_{IndPak}$ - Indian and Pakistani News Channels - Contains English, Romanized and Devanagiri Hindi.
$\mathcal{D}_{ABP}$ - Bengali News Network - English, Romanized Hindi, Romanized and native script Bengali.
$\mathcal{D}_{OTV}$ - Oriya News Network - English, Romanized Hindi, Romanized and native script Oriya.

In all these cases, our model is able to perform exceptionally well. In cases where it makes sense, performance is on par with popular and commercial models. For full details see our paper.

I’m attached the embedding space from $\mathcal{D}_{IndPak}$ here:

Notice how clean the full embedding space is.

Intuition

Why is this happening?

The skipgram training scheme predicts a context for an input word.

A Hindi word’s context is likely to consist of other Hindi words; and example contexts retrieved in the negative sampling phase are likely to be written in languages distinct from the language of the input word.

It just so happens that the skipgram training scheme is perfectly designed to separate out a multilingual corpus into its monolingual components.

And the embeddings reflect this - as we can see from the TSNE plots above.

Note that inside the embedding subspace corresponding to a language, we observe similar properties typically observed when these word embeddings are trained on monolingual corpora.

i.e. standard stuff like analogies, semantic similarities etc. work very well.

In The Wild

We have used this technique in a variety of text analyses centered in South and South-East Asia.

In particular, South Asian social media users used Romanized variants of their native languages that are mostly unsupported by popular language identification systems.

This unsupervised method is highly capable in such situations eliminating significant annotation requirements.

In this paper analyzing the Rohingya crisis, from among dozens of languages (many low-resource), this technique was utilized to extract out English social media posts.

In an analysis of the India-Pakistan crisis of 2019, this technique was used to separate out Romanized Hindi and English for further analysis.

In soon to be published work analyzing the 2019 Indian election, nearly a dozen Indian regional languages were extracted and analyzed with zero annotation burden.

In almost all these cases existing solutions performed poorly or involved prohibitive costs.

For linguistically diverse regions, we foresee that polyglot embeddings are going to be an important NLP pipeline component.

Unsupervised or low-supervision methods have been increasingly deployed in multlingual settings.

For instance, embeddings learned by machine translation models seem to be organized along linguistic lines.

A clever training setup was used for unsupervised machine translation in this paper.

Polyglot embeddings (contextual or otherwise) have been used for performance gains in several tasks.

Cite

@inproceedings{kashmir,
  title={Hope Speech Detection: A Computational Analysis of the Voice of Peace},
  author={Palakodety, Shriphani and KhudaBukhsh, Ashiqur R. and Carbonell, Jaime G},
  booktitle={Proceedings of ECAI 2020},
  pages={To appear},
  year={2020}
}

2019 Sculpture Portfolio

Wed, 29 Jan 2020 06:17:59 UT

The 2019 portfolio:

Paisley hairpin
Rose in maple
Orchid corsage
Paisley in maple
Lily in basswood
Mapley paisley and rest
Foliage in cypress
Mahogany rest
Rococo acanthus in basswood

Interview with The Hindu

Fri, 27 Dec 2019 03:00:13 UT

I recently did an interview with India’s 2nd largest newspaper - The Hindu - about my cartography work with the instagram account @indiaviz.

Do read it and let me know what you feel. I am also soliciting ideas for collaborations and that sort of thing.

Here are the images we used in the interview:

shriphani

city_streets

World Population Distribution - Insights

Sat, 02 Nov 2019 05:40:47 UT

I just finished rendering the Gridded Population of the World (GPW) dataset. I chose the cyan on black color theme since that is a staple of all my cartography projects. Some insights that people pointed out:

Nile River

It is amazing how much of Egypt is just dark and yet there’s this bright along the Nile river where presumably everything - the Pharaohs, Cleopatra, Roman conquest, British colonization, Nasser, and today’s events happened. That is 7000+ years of history right there.

Ganges Plains

The bright spot in Northern India starting at the foot of the Himlayas and spanning the states of Uttar Pradesh, Bihar, West Bengal, and Bangladesh feature several of the most densely populated cities in the world.

Pakistan; Punjab

Pakistan’s population is almost entirely situated around the rivers Indus, Jhelum, Chenab, Ravi, and Sutlej - the 5 rivers are referenced in the name of Punjab.

Australia, New Zealand

The vast dark expanse that is the outback, and then the coastal cities. NZ is actually in the map - social media on certain sites compresses the image to the point where it appears that it isn’t included.

Nigeria, Sub-Saharan Africa

Nigeria is all set to become the third most populous country this century. Sub-saharan Africa is likely to house 1 billion people soon. The region is a hotbed of civil unrest and has seen a dozen or so civil wars in recent history.

Remarks

Were an alien to visit earth and study her dominant species, it would be hard to arrive at the conclusion that mankind originated in East Africa starting from this map alone.

Access to water seems to be the primary driver of human settlement. You can easily trace out the coastlines, islands, and river watersheds from this map.

Feel free to share any other remarks either below or on social media.

Reddit: r/dataisbeautiful, r/mapporn, r/India

Maple Paisley and Rest

Fri, 11 Oct 2019 05:34:36 UT

Maple, aged mahogany.

View this post on Instagram

A post shared by Shriphani Palakodety (@life_of_ess) on Oct 10, 2019 at 5:02pm PDT

Lily in Basswood

Thu, 11 Jul 2019 17:08:03 UT

We just finished this basswood lily for a client. Simple, subtle, and full of life. A glorious amber shine envelops it. A perfect gift for a soulmate.

View this post on Instagram

A post shared by Shriphani Palakodety (@life_of_ess) on Jul 9, 2019 at 6:00pm PDT

Paisley Box

Wed, 19 Sep 2018 02:50:15 UT

I rushed to finish this latest piece for a gallery deadline

I wrapped up that entire piece — cutting, designing and carving in a 2 week period. There is tons of evidence that it was rushed but whatever — I saw it through and that is all that matters.

Nearly a decade ago, a teacher from high school mentioned to me that time was stretchable. When pushed, an individual could accomplish quite a bit with severe time constraints.

This got me thinking about a small NSF experiment from 2 years ago- the NSF dropped deadlines and the number of grant applications halved.

Let us parse that for a second — the most potent fuel for American scientific research is the grant application deadline.

Michelangelo’s Pieta — the work of a grandmaster the likes of which we will never see again- was produced over a 2 year period. Contrast that with the last 2 years of your life.

— —

Here are a few more pictures of the piece.

And a small video illustrating the making of (sorry about the instagram embed)

View this post on Instagram

Paisley jewelry box - the making of video - ð - Always this late - Odesza Model - @baysi_ #wood #woodworking #design #paisley #timelapse #diy #howto #handmade #madeinusa #iran #china #india #arabesque

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on Aug 16, 2018 at 9:00am PDT

Paisley Sculpture 1

Thu, 28 Jun 2018 01:47:46 UT

A new piece - a paisley sculpture carved in relief out of maple and mahogany.

The mahogany pedestal features rosettes and small chip-carved patterns incorporating incisions. The paisley piece rests in a bed of felt.

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on Jun 7, 2018 at 10:01am PDT

Here is a small sampling of the detail work on the pedestal.

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on Jun 7, 2018 at 9:45pm PDT

And a small video of the carving process.

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on Jun 25, 2018 at 3:25pm PDT

Jewelry Tray

Thu, 28 Jun 2018 01:46:19 UT

A small study in paisleys - I’ve been developing a uniquely Persian / Indian form of scroll work incorporating elements from Indian embroidery.

On exhibit at the Bay Area Woodworking Association this month.

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on May 7, 2018 at 10:32am PDT

And here’s a build video:

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on May 14, 2018 at 5:06pm PDT

Mahogany Earrings

Thu, 28 Jun 2018 01:39:12 UT

This project is a pair of wooden earrings. Carved from a single block of mahogany and dealt with from start to finish with hand-tools.

The brass was a stray piece of artist wire combined with some off-the-shelf clip-ons.

An unorthodox set of materials to craft a one-of-a-kind object.

Model: Ivy Zheng

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on May 1, 2018 at 9:58am PDT

Here is a small clip illustrating the whole build process.

A post shared by Shriphani “Ess” Palakodety (@life_of_ess) on May 3, 2018 at 9:02am PDT

Cherry - Purpleheart Spoon

Wed, 28 Mar 2018 19:43:54 UT

For my little cousin I put together this nifty little present - a spoon hand carved from cherry and a small purpleheart stand. I had a lot of fun making this and I could knock this out in a couple of hours - a new milestone for my skills. The improved confidence lets you make aggressive quick strokes and remove large chunks of material - where previously one would be as conservative as possible.

Acanthus Leaf - Black Walnut

Thu, 01 Mar 2018 07:37:41 UT

This is a project very close to my heart. I first saw the acanthus leaf perched at the top of columns in old buildings in London. There’s something about the acanthus leaf that makes it stand out - the incredible fractality throughout.

I had a lot of fun carving this. And carving such an old ornament (found in sculpture as early as 1000 BC), you can’t help but feel humbled – 3000 odd years ago, someone like me was making the same thing. What were their dreams like? What drove them and their creative pursuits?

Here’s the leaf in her full glory - the chosen ornament of Western civilization, the muse of the great architect Callimachus, the artistic vision for over a hundred generations of mankind - acanthus.

Music is my main source of inspiration. Music guide the curves and shapes in what I carve. Arvo Pärt’s Spiegel im Spiegel seemed like the most natural fit. The curve of the leaf isn’t too severe or too mild, the shape not too exxagerated, the grain not too overwhelming to show the cuts.

A post shared by Shriphani Palakodety (@life_of_ess) on Feb 27, 2018 at 10:17am PST

Mahogany Paisley Carving

Tue, 02 Jan 2018 02:47:13 UT

I carved this out of a slab of mahogany. The paisley shape originated in Persia, was adopted heavily in Indian design, is a regular feature in henna and other art forms, and was adopted heavily by the print industry in Glasgow.

For this couple that contributed heavily to my life and my mental health, this was my way of saying thanks.

I was operating on a very tight deadline so I couldn’t clean things up as much as I wanted.

As we say, the next piece will always be better.

Here’s a video of the carving process.

Time lapse video of a mahogany paisley carving. #mahogany #wood #woodworking #woodwork #carving #sculpture #timelapse #handmade #handcrafted #art #artist #artistsoninstagram #scroll #artprocess #indian #persian #british #indianwedding

A post shared by Shriphani Palakodety (@life_of_ess) on Dec 29, 2017 at 1:36pm PST

Cedar and Cherry Chopsticks

Tue, 02 Jan 2018 02:28:01 UT

Made for a friend’s birthday, these chopsticks are made from cherry and the base from spanish cedar.

I began incorporating a new tool in my craft - a spokeshave. Amazing tool.

Basswood Carving

Tue, 02 Jan 2018 02:08:07 UT

A basswood carving. Finished first with polyurethane, which was a disaster. Rescued (sort-of) with a gel stain.

Very proud to show you guys the finished piece. Acanthus hand carved on basswood and finished with a gel stain and lacquer. She looks like a chocolate bar - and the glossy nature makes her look menacing - a little like a panther. This piece was a little ruined by my lack of experience but someone I respect a lot forced me to stick with it and see it through - these folks are God’s gift to men - they help us see and think straight when we need it the most. #wood #woodworking #art #artistsoninstagram #artprocess #artist #carving #handmade #woodcarving #crafts #chocolate #fractal #artoftheday #glossy #shine #plant #trees #craftsmanship #sculpture #sculpting #woodsculpture

A post shared by Shriphani Palakodety (@life_of_ess) on Nov 4, 2017 at 4:30pm PDT

New woodworking project WIP. Hand carved acanthus plant on basswood. Sped up about 500x - this condenses parts of a 30+ hour project and a new addition to my woodworking skillset. #art #artprocess #artist #artistsoninstagram #artwork #woodworking #wood #carving #woodcarving #handtools #handmade #handcarved #timelapse #hyperlapse #carpenter #carpentry #diy #sculpture #sculpting #basswood #woodcraft #woodworkeroninsta #realwoodworking #carpenters #chisel #artisanal #artisan

A post shared by Shriphani Palakodety (@life_of_ess) on Nov 3, 2017 at 6:38pm PDT

Kitchen Tools - Spatula And Serving Board

Tue, 02 Jan 2018 02:00:00 UT

I want to share a couple more woodworking projects.

The first is a jatoba spatula, the second a bloodwood cutting board.

Jatoba is a stunning wood - after sanding down to 1500 grit, when you run your finger on the surface, it has this velvety, posh, expensive feel.

It feels gorgeous to look at, it is amazing to touch.

The bloodwood board was rescued from a scrap bin and cleaned up.

Removing the dirt and grime revealed a surface full of character and vibrant colors. The contrast between the live edge and the grain, the reds, the blacks and the yellows - this is a masterpiece forged by nature - one no man can produce.

SHRIPHANI PALAKODETY: SHRIPHANI PALAKODETY

Indic Language Identification

Geographic Extents

Bi / Multi Linguality

Links

Richmond Carvers

Maple Panel

Kitchen Tools

Polyglot Word Embeddings Discover Language Clusters

Background

Polyglot Embeddings

A Language Identification System

How Good Are These Clusters?

Intuition

In The Wild

Links

Cite

2019 Sculpture Portfolio

Interview with The Hindu

World Population Distribution - Insights

Nile River

Ganges Plains

Pakistan; Punjab

Australia, New Zealand

Nigeria, Sub-Saharan Africa

Remarks

Maple Paisley and Rest

Lily in Basswood

Paisley Box

Paisley Sculpture 1

Jewelry Tray

Mahogany Earrings

Cherry - Purpleheart Spoon

Acanthus Leaf - Black Walnut

Mahogany Paisley Carving

Cedar and Cherry Chopsticks

Basswood Carving

Kitchen Tools - Spatula And Serving Board

SHRIPHANI PALAKODETY: SHRIPHANI PALAKODETY

Indic Language Identification

Geographic Extents

Bi / Multi Linguality

Links

Richmond Carvers

Maple Panel

Kitchen Tools

Polyglot Word Embeddings Discover Language Clusters

Background

Polyglot Embeddings

A Language Identification System

How Good Are These Clusters?

Intuition

In The Wild

In Related News

Links

Cite

2019 Sculpture Portfolio

Interview with The Hindu

World Population Distribution - Insights

Nile River

Ganges Plains

Pakistan; Punjab

Australia, New Zealand

Nigeria, Sub-Saharan Africa

Remarks

Discussions on Social Media

Maple Paisley and Rest

Lily in Basswood

Paisley Box

Paisley Sculpture 1

Jewelry Tray

Mahogany Earrings

Cherry - Purpleheart Spoon

Acanthus Leaf - Black Walnut

Mahogany Paisley Carving

Cedar and Cherry Chopsticks

Basswood Carving

Kitchen Tools - Spatula And Serving Board