In this article, you will learn about Excel’s dynamic array formulas and how to use them with the help of simple examples.

What Is An Array?

What precisely is an array? In Excel terms, an array is a collection of values, whether they are in rows, columns, or both. 

An array formula allows us to do multiple calculations on one or more values that are stored in our array. 

These formulas update automatically when the referenced cells change. And that is why they are called dynamic arrays. 

Dynamic arrays perform automatic calculations, and they return values to one cell or multiple cells based on our formula. 

To do this, we need to input the formula, usually in just one cell.

Note: This is a guest post by Ben Richardson. Ben is an ex-banker and venture capitalist. He now runs Acuity Training, one of the UK’s leading providers of Excel training courses.

Short History Of Dynamic Arrays In Excel

In previous versions of Excel, dynamic arrays were complex to set up. 

You would create an array using the keystroke combination CTRL + SHIFT + ENTER, thus often referred as CSE formulas.

With the introduction of dynamic arrays formulas in September 2018, Microsoft introduced (in Office 365) a more accessible approach to creating them. 

When we create such formulas, they are automatically populated or spilled into the corresponding adjacent cells. If Excel does not identify them for some reason, we can still use CTRL + SHIFT + ENTER. 

Suppose you write a dynamic formula in Excel. In that case, it can now recognize if this formula can return multiple values or not.

Using Dynamic Arrays In An Example

Suppose that we have salaries for an average company for four years:

We can now tell Excel that the first two numbers (B2:C2) are a part of an array. To do this, we will place the following formula in cell B5:

=B2:C2

Before you press ENTER look at the formula bar.

Suppose you see that your formula is enclosed in curly brackets. In that case, it is a sign that Excel identifies it as an array formula.

Once you hit ENTER, the array will be “spilled” horizontally. 

This means that Excel effectively duplicates the formula for each cell in the array.

A quick way of knowing that this is now an array is that we have the cells formatted with blue borders:

You can see that there is nothing scary about the arrays, right? 

Now, suppose that we have the number of employees in the company for every year as well:

Our goal now is to find out the average salary for every year. You could go along and input the formula cell B4:

=B2/B3

Then drag this formula to the end of our table. 

You would have the following results:

To avoid so many steps in a process, we can use a Dynamic Array. 

To do this, we will enter the formula in cell G2:

=B2:E2/B3:E3

Excel will automatically recognize the pattern and will do the calculations for us:

You can see that cells H2:J2 now have a blue outline, meaning that the formula spilled across multiple cells.

Of course, if we change any value in our original table, the resulting numbers in the array will also change. 

You may still need to correct the formatting of the results.

Excel automatically uses the General formatting style for numbers.

The example above worked on multiple cells. 

We can also use dynamic arrays with single-cell array formulas. 

For example, suppose you want to know the difference between $1,000 and the average salary. In that case, we could use the following formula:

=MIN(1000-B4:E4)

We would get the number $37.57 for the result:

Before dynamic array formulas, the usual process would have been to subtract every number in the fourth row by 1,000 and then find the minimum value of these values with the MIN formula, a complicated task.

Dynamic Array Functions

Let’s see some common de

With the development of dynamic arrays and their full integration into Excel with Office 365, Microsoft has recently added dynamic array functions to Excel.

These functions are:

UNIQUE – we can use it to extract unique values from our range.

FILTER – used to filter our data based on any criteria we specify.

SORT – sorts our range by the desired column.

SORTBY – sorts our range by another array or range.

RANDARRAY – used to present an array of random numbers.

SEQUENCE – used to present a list of sequential numbers.

Conclusion

Next time you are laboriously copying a formula across a table in Excel, pause and check whether you get the same results with a dynamic array formula.

Dynamic arrays can take a little getting used to, but you’ll never go back once you do — they are big timesavers and worth the time to master.

The post An Introduction To Dynamic Array Formulas In Excel appeared first on nandeshwar.info.

Artificial Intelligence (AI) has two mechanisms of learning: Machine Learning and Deep Learning. As these two phrases are often interchangeable, let’s discover the differences between them in this article.

Note: This is a guest post by author Anita Basa, an enthusiastic Digital Marketer and content writer working at Tekslate.com. I write articles on trending IT-related topics such as Microsoft Dynamics CRM, Oracle, Salesforce, Cloud Technologies, Business Tools, and Software. You can find me on Linkedin: Anita Basa

Machine Learning

Machine learning uses the information to instruct and search for accurate results. Its focal point is creating software that can easily access information and learn from the data. Many algorithms within machine learning support the classification or prediction of outputs given some input data.

Deep Learning

Deep learning is a subset of Machine Learning, but artificial neural networks are the primary algorithms for training models. Researchers modeled artificial neural networks after the human brain to connect the different data inputs (aka nodes) through synapses. While in typical neural networks, the number of layers between the input and the output is few, the layers can grow in hundreds and more in deep learning models.

Kinds of Deep Learning Algorithms

Let’s see some common deep learning algorithms.

Recurrent Neural Networks:

The Recurrent Neural Networks use a critical component that is not available in simpler algorithms: memory. The computer can remember past data and decisions in memory and consider them when evaluating current data and introduce the power of context.

Researchers use Recurrent Neural Networks for NLP (natural language processing) work because the computer understands a text better if it can access memory of tone and context.

Convolutional Neural Networks:

Researchers use Convolutional Neural Networks to work with pictures. The term convolutional is for the technique that applies a weight-based filter throughout each detail of an image, assisting the computer to recognize and react to elements in the image itself.

The technique is beneficial when you must scan a high-resolution image for a particular product or feature. A specialized field of studying photo data is called computer vision — and is a growing field.

Difference between Deep Learning and Machine Learning

After learning the basics of the machine and deep learning, here are some crucial points of machine learning vs. deep learning

Feature Engineering

Feature Engineering is a process of using domain knowledge to build features or inputs for training models. Data scientists use feature engineering to transform existing data to increase the accuracy of their learning models. It is a time-consuming process and requires knowledge of the data and techniques.

In Machine Learning, mainly the applied features must be analyzed by an expert and then hand-coded as per the domain and data type. For instance, the components can be shape, texture, pixel value, orientation, and position. The work of machine learning algorithms depends on how exactly the elements are identified and extracted.

In Deep Learning, the algorithms are trying to learn advanced features from data. It helps lessen the work of creating a new feature extractor for every problem. 

Data Province

A principal distinction between deep learning and machine learning is its overall performance as the size of statics increases.

In Deep learning, if the information quantity is small, it does not perform well because the deep learning method requires a considerable quantity of data to comprehend it accurately. 

The Machine Learning methods with their own designed rules will survive in this situation quickly and efficiently. 

Interpretability

We use interpretability as a feature for comparison between machine and deep learning. 

If we use deep learning to provide self–regulating scores to essays, its performance in scoring is impressive, and it is near-human performance. But there is a difficulty; it does not disclose the behind-the-scenes details or answer questions like why it has given that score. However, mathematically you can discover which nodes of the deep neural network are activated. Still, we don’t know what their neurons were supposed to copy and what these layers of neurons are performing collectively. 

The Machine learning algorithms such as decision trees give you crisp rules as to why it is selected, making it easy to interpret the reason behind it. Moreover, the algorithms like decision trees and logistic regression are used in heavily regulated businesses for interpretability.

Equipment Province

Deep Learning methods are mainly dependent on exclusive devices compared to old machine learning methods that are worked on backend machines. It occurs because the demand for deep learning algorithms includes GPUs that are an internal process of working.

Deep Learning methods naturally perform a vast quantity of matrix multiplication processes. These functions could effectively optimize by using GPU because GPU is specifically created with this goal.

Execution Time 

Usually, deep learning algorithms take longer to train because there are various deep learning algorithms that provide training, which takes longer than usual. State-of-the-art deep learning algorithm ResNet is required two weeks to train entirely from scratch. However, the Machine Learning Algorithms take less time to prepare, ranging from a few minutes to a few hours. 

The testing or validation time, however, is the opposite. Deep learning algorithms take less time to perform the testing process. However, suppose you compare it with k – nearest neighbor, a type of machine learning algorithm. In that case, the test time increases with the increasing data size even though it does not apply to all machine learning algorithms, as some take less time to run.  

Brainstorming Method

When solving the problem using machine learning algorithms, it is advisable to break them into various parts, resolve them specifically, and mix them to obtain results. In contrast, deep learning advocates solving the difficulty at the backend.

For example, if you have work for dual-target detection, it is to search the target and where it is presented in the photo.

In the Machine learning method, you can split the issue into target detection and target recognition. Firstly you use a complete pack detection method like grab cut to remove images and search required targets. Then all the identified things, you will use object detection methods like SVM with HOG to identify pertinent targets.

In the deep learning method, you would follow the procedure at the backend. In a Yolo net (it is a kind of deep learning method), you will present a photo, and it will provide you with the location and the title of the object. 

Conclusion

Deep learning is an advanced machine learning system that relies on often unstructured and enormous quantities of data. Thus, deep learning can cater to a bigger cap of issues with extra ease and efficiency. Technological breakthroughs like Google’s Deep Mind are the epitome of present-day AI, facilitated by deep learning and neurological networks.

The post Machine Learning vs. Deep Learning – What is the Difference? appeared first on nandeshwar.info.

We will see three ways of creating PowerPoint presentations:

1. Using the default RMarkdown option
2. Using a custom PowerPoint template
3. Using officedown/officer R libraries for further customization

Before we jump into creating these presentations, I would like to share a quick overview of PowerPoint and presentation design best practices from two books: Presentation Zen and Strategic Storytelling.

Presentation Zen

Garr Reynolds, the author of Presentation Zen, recommends the use of full-bleed background images with one sentence, a word, or statistic. He also recommends using the rule of thirds to place the point of focus in the intersection of lines dividing the slide into three equal sections horizontally and vertically. We will see whether we can make that happen. He also suggests using simple charts to make your point. Although these types of presentations are better for keynotes, we will try to apply some of these principles.

Strategic Storytelling

Dave McKinsey, the author of Strategic Storytelling, provides examples from top management consulting companies such as McKinsey and Accenture. He explains why these business presentations work. Some of the main points from this book are:

1. Use of the Minto principle
2. Use the slide title to make your point or ask a question
3. Use of body content to support your point mentioned in the title

Let’s talk about the Minto principle also called the Minto Pyramid.

Barbara Minto was McKinsey consulting’s first female MBA professional. She invented the Minto pyramid to help consultants make effective business presentations. It provides a good structure for organizing your presentation content. You start with your main idea or recommendation. Then provide the supporting arguments with data and facts. Each supporting argument is further supported by more arguments and data. If you notice any of the consulting company’s presentations, you will see they have many charts, call-outs, and notes with references. 

With these presentation design best practices, let’s dive into creating PowerPoint presentations with R and RMarkdown.

For this post, let’s say that we are studying the streaming services market. Our client is a company trying to enter in this market.

Here’s how we will structure the presentation:

• Slide 2: Our main recommendation.

• Slide 3: Survey of the market: share prices

• Slide 4: Survey of the market: market share

• Slide 5: Opportunities

• Slide 6: Side by side comparison of Netflix and Disney+

• Slide 7: Conclusion

Let’s get started.

First, let’s make sure you can build the example PowerPoint presentation in RStudio.

To start a new markdown document, click on File, New File, R Markdown, then Presentation, and finally PowerPoint. 

The title is of my presentation is “Streaming Services Market Study” and the author is “Best Brians Consulting” (It is Brians and not brains after the movie Life of Brian)

Make sure that this document compiles and gives you a nice PowerPoint presentation. 

Click on the “knit” button. Save your file. If everything works out, you should have a new PowerPoint presentation. 

If everything worked out, edit the RMarkdown document with the new content for this streaming services presentation.

First, let’s change the knitr options to cache the data and suppress warnings and messages.

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = FALSE, cache = TRUE, warning = FALSE, message = FALSE)
```
Code language: R (r)

The first slide has recommendations. 

## Recommendations
- This is a tough market with many established companies
- You will need a unique strategy to set yourself apart
- We recommend you enter in a narrow market and not compete with the big companies
Code language: R (r)

Our next slide has data with the share prices of the companies in this space.

Following the “Strategic Storytelling” book example, the title of our slide is: Companies such as Netflix have cemented their roles as streaming entertainment providers

This R code pulls the historical share prices of Netflix, Disney, and Comcast, ATT, and shows a ggplot multiple line plot. It could be made prettier and more valuable with annotations, but for this exercise, it is good enough.

## Companies such as Netflix have cemented their roles as streaming entertainment providers

```{r sharepricesplot, fig.asp=0.6, dpi=300}
library(tidyquant)
library(lubridate)
library(ggplot2)
library(ggthemes)
library(dplyr)

my_tickers <- c("NFLX", "DIS", "T", "CMCSA")

stock_prices <- tq_get(x = my_tickers,
                       from = Sys.Date() %m-% years(5),
                       to = Sys.Date(),
                       get = "stock.prices")

g <- ggplot(stock_prices,
            aes(x = date, 
                y = close, 
                group = symbol, 
                color = symbol,
                label = symbol)) +
  geom_line()


label_data <- tribble(
  ~date,       ~symbol, ~close,
  as.Date("2019-01-02"), 'NFLX', 220,
  as.Date("2019-01-02"), 'DIS', 140,
  as.Date("2019-01-02"), 'CMCSA', 48,
  as.Date("2021-01-02"), 'T', 40,
  
)

g + 
  theme_wsj() +
  geom_text(data = label_data, aes(x = date, y = close, label = symbol, color = symbol)) +
  ggtitle(label = "Daily Closing Share Prices") +
  theme(legend.position = "none", 
        title = element_text(size = rel(1.3), family = "Arial"),
        plot.title.position = "plot") 

```
Code language: R (r)

Our next slide is the market share data.

This R code creates a treemap of market share data using ggplot and treemapify. This chart could also be made better, but this will do for now.

## The market is packed with many streaming options

```{r marketshare, fig.asp=0.6, dpi=300}
market_share <- tribble(
  ~company, ~share,
  "Netflix", 0.2,
  "Amazon Prime", 0.16,
  "Hulu", 0.13,
  "Max", 0.12,
  "Disney", 0.11,
  "Apple", 0.05,
  "Peacock", 0.05,
  "ESPN", 0.04,
  "Starz", 0.03,
  "Showtime", 0.03,
  "Paramount", 0.03,
  "YouTube TV", 0.01,
  "Sling TV", 0.01,
  "BritBox", 0.01,
  "Other", 0.02,
)

library(treemapify)
library(scales)

ggplot(market_share, aes(area = share, fill = company, label = paste(company, percent(share), sep = "\n"))) +
  geom_treemap() +
  geom_treemap_text(color = "black", 
                    place = "center",
                    grow = F) +
  scale_fill_brewer(palette = "Set3") +
  labs(title = "Market Share of Streaming Services", 
       caption = "Source: The Wrap. https://www.thewrap.com/netflix-streaming-us-market-share-chart") + 
  theme(legend.position = "none", 
        plot.title = element_text(face = "bold"),
        plot.caption = element_text(hjust = 0),
        plot.title.position = "plot", 
        plot.caption.position = "plot")
```
Code language: R (r)

The next slide is a chart from a website showing opportunities. I have used include_graphics from knitr to add the image. I have also added a caption by using fig.cap option.

## ... but it is possible to pull some customers away

```{r streamingservicescomparision, fig.cap="Although Netflix still is the market leader, it dropped from 29% to 20% from 2020 to 2021. Source: https://www.thewrap.com/netflix-streaming-us-market-share-chart"}
knitr::include_graphics(path = "https://www.thewrap.com/wp-content/uploads/2021/03/040221-U.S.-Streaming-Market-Share-2020-versus-2021.png")
```
Code language: R (r)

Next up is a case study of Disney+ vs. Netflix. I am using a two-column layout using a special Pandoc syntax. Within each column, I have a column header and a bulleted list. I also added the source in the first column.

## A case in point: Disney+

:::::: {.columns}
::: {.column}
Netflix

- 200M Subscribers
- Started 13 years ago
- Projected to get 300M subscribers

Source: cnet
:::

::: {.column}
Disney+

- 95M Subscribers
- Started 15 months ago
- Projected to reach 250M subscribers
:::

::::::
Code language: R (r)

Lastly, we have the conclusion slide with one sentence. 

## Conclusion

This is a tough market to break into, but you can carve a niche out. 

Don't go head-to-head with the big companies.
Code language: R (r)

After hitting knit, you will see that the PowerPoint presentation has the line plot and the treemap we generated using ggplot.

It also has an external image from a URL.

It created the two-column layout as well.

But you notice some other problems too.

• Page numbers are missing.
• The aspect ratio is not 16 by 9.
• The slide titles are aligned in the center and the long title doesn’t fit neatly.
• Both the charts are taking the same smallish space.
• The conclusion slide looks bland.

We can resolve some of these issues by providing our custom PowerPoint template. 

1. Create a new PowerPoint presentation. 
2. Then edit the layouts. 
3. Go to View and click on Slide Master.

Here you will see all the default layout options.

RMarkdown and Pandoc currently support only these four layouts:

• title
• title and content
• section header, and
• two-column layout

That means we can customize only these four layouts. 

1. Edit the master slide first.
2. Click on the “click to edit” text box.
3. Change the font type to your liking. I selected “Lato” for the title.
4. Next, I am selected Corbel for the body or content area.
5. Add custom text boxes in the footer for date and page numbers.
6. Insert the date and slide number from the Insert menu.
7. Change the font type and size.
8. Then delete all the existing boxes for footers.
9. I added a logo in the center of the slide. I created this simple logo using Logomakr.com

Let’s also change the background color of the slides to something light. I chose #edf6f9 as the background color.

1. Go to Background Styles. 
2. Format Background. 
3. Click on Color and then more colors.

Let’s also change the color of the slide title. I chose #006d77.

Save the PowerPoint template. Add a reference in the YAML in your RMarkdown to this PowerPoint template like this:

output:
  powerpoint_presentation:
    reference_doc: template.pptx
Code language: R (r)

Hit the “knit” button to see the results.

You can change the font type of the plots to match your presentation font type by using the “showtext” library. But I leave that for you.

If you have a simple presentation, this can work very well. You pull all your data in one place, create your charts, and add recommendations. You can edit the PowerPoint later, of course, but for repeatability, we want to keep as many things in RMarkdown as possible.

One disadvantage of this approach using Pandoc is that we are limited to only four layouts. If you wanted to add one title and one subtitle to your layout, it won’t work. Another disadvantage is an image or a table can’t exist with other content. If you add content to a slide that already has a chart or an image, it will get pushed to a new slide.

You can get around this problem using patchwork or similar libraries to place side-by-side graphs, tables, and text. But that may not give you the best resolution. We also can’t add random texts or call-outs.

There’s an option to add background images in HTML presentations, but it doesn’t work for PowerPoint.

Let’s explore the office and officedown libraries for further customization. You can use the officedown in YAML as officedown::rpptx_document and provide a template, but I don’t know the exact advantages of doing so.

Instead, we will create an R file and build the PowerPoint presentation one slide at a time.

Let’s load the officer and magrittr libraries.

library(officer)
library(magrittr)
Code language: R (r)

Next, we will load our template file using the read_pptx function in a variable called my_ppt.

my_ppt <- read_pptx(path = "template.pptx")
Code language: R (r)

With the layout_summary function, we can check all the layouts available to us.

officer::layout_summary(my_ppt)
Code language: R (r)

Add a new layout to your PowerPoint template with one text box and increase the font size to 80. Rename this layout to ‘one big number’.

Save the template and read the file again.

Now when you run layout_summary you will see that this new layout is available to us.

The layout_properties function lets us see the available content areas we can change.

layout_properties(my_ppt, layout ="Title Slide")
Code language: R (r)

This is an important function. Notice the `ph type` argument value. We need to use the value of `ph type` to set the content.

It will be easier with an example.

my_ppt <- add_slide(my_ppt, 
                    layout = "Title Slide", 
                    master = "Office Theme") %>%
  ph_with(value = "Streaming Services Market Study", 
          location = ph_location_type(type = "ctrTitle")) %>%
  ph_with(value = "Best Brians Consulting", 
          location = ph_location_type(type = "subTitle"))
Code language: R (r)

Let’s create our first slide with the presentation title slide.

To add a slide we use add_slide function. To this function, we specify the layout and the theme to use.

To add content we use the ph_with function. The value argument of this function specifies the content.

And now the important part: where do we want the content to go? To find the location, we use ph_location_type function and pass the type we saw in ph type.

For the opening, title slide the box is called ctrTitle.

Let’s add our title.

Also, add the author to ph type of subtitle.

Let’s save this new presentation using the print command.

print(my_ppt, target = "officer-presentation.pptx") 
Code language: R (r)

Open the PowerPoint and check whether you can see the title and subtitle.

Let’s add the second slide with recommendations. But check the available shape types using the layout_properties function for the Title and Content layout. You will that we have a title and body types available.

Let’s add ‘recommendations’ as the title and pass the bullet points for this slide as a vector to the body location.

my_ppt <- add_slide(my_ppt, 
                    layout = "Title and Content", 
                    master = "Office Theme") %>%
  ph_with(value = "Recommendations", 
          location = ph_location_type(type = "title")) %>%
  ph_with(value = c("This is a tough market with many established companies",
                    "You will need a unique strategy to set yourself apart",
                    "We recommend you enter in a narrow market and not compete with the big companies"), 
          location = ph_location_type(type = "body"))
Code language: R (r)

Save the document again and see what it looks like.

Next, add a slide with the share prices chart, but we need to create this chart first. This code will save the chart in a ggplot object called share_prices_plot.

my_ppt <- add_slide(my_ppt, 
                    layout = "Title and Content", 
library(tidyquant)
library(lubridate)
library(ggplot2)
library(ggthemes)
library(dplyr)

my_tickers <- c("NFLX", "DIS", "T", "CMCSA")

stock_prices <- tq_get(x = my_tickers,
                       from = Sys.Date() %m-% years(5),
                       to = Sys.Date(),
                       get = "stock.prices")

g <- ggplot(stock_prices,
            aes(x = date, 
                y = close, 
                group = symbol, 
                color = symbol,
                label = symbol)) +
  geom_line()


label_data <- tribble(
  ~date,       ~symbol, ~close,
  as.Date("2019-01-02"), 'NFLX', 220,
  as.Date("2019-01-02"), 'DIS', 140,
  as.Date("2019-01-02"), 'CMCSA', 48,
  as.Date("2021-01-02"), 'T', 40,
  
)

share_prices_plot <- g + 
  theme_wsj() +
  geom_text(data = label_data, aes(x = date, y = close, label = symbol, color = symbol)) +
  ggtitle(label = "Daily Closing Share Prices") +
  theme(legend.position = "none", 
        title = element_text(size = rel(1.3), family = "Arial"),
        plot.title.position = "plot") 
Code language: R (r)

We will again use title and body locations to add this plot and the title. The only difference is that we can provide a resolution of the chart.

my_ppt <- add_slide(my_ppt, 
                    layout = "Title and Content", 
                    master = "Office Theme") %>%
  ph_with(value = "Companies such as Netflix have cemented their roles as streaming entertainment providers", 
          location = ph_location_type(type = "title")) %>%
  ph_with(value =  share_prices_plot, 
          location = ph_location_type(type = "body"),
          res = 300)
Code language: R (r)

Next, let’s add the tree map with the market share

market_share <- tribble(
  ~company, ~share,
  "Netflix", 0.2,
  "Amazon Prime", 0.16,
  "Hulu", 0.13,
  "Max", 0.12,
  "Disney", 0.11,
  "Apple", 0.05,
  "Peacock", 0.05,
  "ESPN", 0.04,
  "Starz", 0.03,
  "Showtime", 0.03,
  "Paramount", 0.03,
  "YouTube TV", 0.01,
  "Sling TV", 0.01,
  "BritBox", 0.01,
  "Other", 0.02,
)

library(treemapify)
library(scales)

market_share_tree_map <- ggplot(market_share, aes(area = share, fill = company, label = paste(company, percent(share), sep = "\n"))) +
  geom_treemap() +
  geom_treemap_text(color = "black", 
                    place = "center",
                    grow = F) +
  scale_fill_brewer(palette = "Set3") +
  labs(title = "Market Share of Streaming Services", 
       caption = "Source: The Wrap. https://www.thewrap.com/netflix-streaming-us-market-share-chart") + 
  theme(legend.position = "none", 
        plot.title = element_text(face = "bold"),
        plot.caption = element_text(hjust = 0),
        plot.title.position = "plot", 
        plot.caption.position = "plot")
Code language: R (r)

my_ppt <- add_slide(my_ppt, 
                    layout = "Title and Content", 
                    master = "Office Theme") %>%
  ph_with(value = "The market is packed with many streaming options", 
          location = ph_location_type(type = "title")) %>%
  ph_with(value =  market_share_tree_map, 
          location = ph_location_type(type = "body"),
          res = 300)
Code language: R (r)

With the default RMarkdown/Pandoc option, we can place an image and text side-by-side, but let’s try that with the officer package and the ‘Picture with Caption’ layout.

We can see we have body, title, and pic locations available for us with the ‘Picture with Caption’ layout.

Let’s add the year-over-year market share comparison chart from the wrap in the pic location and some text in the body location.

ph_wiyoy_market_comp_img <- file.path("wrap-yoy-market-share.png")

my_ppt <- add_slide(my_ppt, 
                    layout = "Picture with Caption", 
                    master = "Office Theme") %>%
  ph_with(value = "... but it is possible to pull some customers away", 
          location = ph_location_type(type = "title")) %>%
  ph_with(value =  external_img(yoy_market_comp_img), 
          location = ph_location_type(type = "pic")) %>%
  ph_with(value =  "Although Netflix still is the market leader, it dropped from 29% to 20% from 2020 to 2021. Source: https://www.thewrap.com/netflix-streaming-us-market-share-chart", 
          location = ph_location_type(type = "body"))
Code language: R (r)

Let’s add and save the file again.

I don’t like the location of the source text. I want to move it close to the footer.

To do so, I need to first create a style or properties I want to apply to this text.

We will use the fp_text function to make the text color light gray and set the font size to 8.

We will name this variable source_box_style. 

source_box_style <- fp_text(bold = FALSE, color = "gray50", font.size = 8)
Code language: R (r)

To add a text box at any location, we need to provide the text box location using the left, top, and other coordinates to the ph_location function. 

source_box_location <- ph_location(
  left = 1,
  top = 6.6,
  height = 0.3,
  width = 11,
  newlabel = "source_box"
)
Code language: R (r)

And to get the text box to show up on the slide, we need to use the fpar function, which creates a paragraph object for PowerPoint. In fpar, we will wrap our source text and provide the style property we created earlier with the ftext function. 

source_box_location <- ph_location(
  left = 1,
  top = 6.6,
  height = 0.3,
my_ppt <- add_slide(my_ppt, 
                    layout = "Picture with Caption", 
                    master = "Office Theme") %>%
  ph_with(value = "... but it is possible to pull some customers away", 
          location = ph_location_type(type = "title")) %>%
  ph_with(value =  external_img(yoy_market_comp_img), 
          location = ph_location_type(type = "pic")) %>%
  ph_with(value =  "Although Netflix still is the market leader, it dropped from 29% to 20% from 2020 to 2021.", 
          location = ph_location_type(type = "body")) %>% 
  ph_with(value =  fpar(ftext(text = "Source: https://www.thewrap.com/netflix-streaming-us-market-share-chart", 
                         prop = source_box_style)),
          location = source_box_location)
Code language: R (r)

Here are the steps again to create a text box:

1. Define style or properties
2. Decide and provide the coordinates to ph_location function
3. Format the given text using the style and ftext function, and
4. finally wrap the ftext call in fpar function to create a paragraph object

You can find the location coordinates by creating a box in PowerPoint and adjusting it. To make it easier, I saved the location object in a variable called source_box_location. 

Save the PowerPoint presentation again. 

Using this same principle, let’s create a callout box and add it to the share prices slide. I created callout_box_style and callout_box_location.

callout_box_style <- fp_text(bold = TRUE, color = "black", font.size = 16, font.family = "Corbel")

callout_box_location <- ph_location(
  left = 2.6,
  top = 3,
  height = 2,
  width = 2,
  newlabel = "call_out_box"
)
Code language: R (r)

To select the slide with share prices, we need to use “on_slide” function and provide the slide number. 

my_ppt <- on_slide(my_ppt, index = 4) %>%
  ph_with(value =  fpar(ftext(text = "Netflix had some stumbles but it is going up steadily", 
                              prop = callout_box_style)),
          location = callout_box_location)
Code language: R (r)

Make this change and then save the presentation again.

print(my_ppt, target = "officer-presentation.pptx")
Code language: R (r)

Of course, it would be easier to draw the text box later in the PowerPoint, but this gives an additional tool to place text boxes with dynamic content at our chosen locations. 

For the final addition, I would like to add a background image that bleeds into the slide and add some text on it. We will use the one big number layout we created earlier. 

Let’s check the layout properties. We see the property name as “one big number.”

I downloaded a photo from unsplash. I named it tv-3.jpg, whose path I am storing in bg_image_path variable. 

bg_image_path <- file.path("tv-3.jpg")
Code language: R (r)

Next, let’s add a slide using the add_slide function and “one big number” as the layout. 

Then we will use the ph_with function to place the external image and 13.38 as the width and height. We will use the ph_location_fullsize function to return the proper dimensions from the slide layout. 

Lastly, we will use another ph_with function to add the text of “31% drop” at the text placeholder location. 

bmy_ppt <- add_slide(my_ppt, 
                    layout = "one big number", 
                    master = "Office Theme") %>%
  ph_with(value =  external_img(bg_image_path, width = 13.38, height = 13.38), 
         location = ph_location_fullsize(),
          use_loc_size = FALSE) %>% 
  ph_with(value = "31% drop",
          location = ph_location_type(type = "body"))

print(my_ppt, target = "officer-presentation.pptx")
Code language: R (r)

These steps should give you a good-looking PowerPoint presentation using RMarkdown. Here’s the summary of our process:

• We created a PowerPoint presentation using the default knitr options in RMarkdown. It worked well but didn’t offer customization. 
• Therefore, we created a PowerPoint template to make the presentation look better, and it did!
• But for more control, we looked at the officer R package and created our layouts, and placed objects where we wanted them.

I hope this helps you when you are trying to create PowerPoint presentations using R and RMarkdown. Let me know what you think by leaving a comment. 

The post Create PowerPoint Presentations with R and RMarkdown appeared first on nandeshwar.info.

Best books on data visualization is a bold claim, but my best doesn't have to be someone else's best. I have benefitted and learned from these books, and you would enjoy reading them too.

Visualizing Data

Although it is an old book, its concepts are still relevant today. William Cleveland, a professor of Statistics and Computer Science at Purdue University, provides great examples of using simple charts and exercising caution with aspect ratios. It is written for an academic audience, but it is hardly intimidating to practitioners. The best part: you can find the R code used for creating the charts from this book.

The Visual Display of Quantitative Information, 2nd Ed.

A data visualization book list will remain incomplete if missing Tufte's books. Edward Tufte, with his beautiful design of the book and charts, has led the charge against bad data visualization a.k.a chart junk.

Show Me the Numbers: Designing Tables and Graphs to Enlighten

When I first started reading about dashboards and good chart design, I bumped into Stephen Few's book Information Dashboard Design. This book changed how I looked at charts and tables. Show me the numbers is a great book if you haven't read any of Few's works. 

Storytelling with Data: A Data Visualization Guide for Business Professionals

Cole Kanflic's Storytelling with data is a great addition to the collection of data visualization books. It is beginner-friendly and does not assume the knowledge of any programming. Most, if not all, the charts in the book can be created in Excel. The result is pleasing to the eyes and is clutter-free.

Good Charts: The HBR Guide to Making Smarter, More Persuasive Data Visualizations

The charts in this book have more of an infographic-y feel to them -- but that's not a negative. I have advocated for simplicity and clarity and criticized "interesting" things, but often you need to splash the charts with some design elements to gain attention. This book provides you great examples of such charts.

The Wall Street Journal Guide to Information Graphics: The Dos and Don'ts of Presenting Data,...

Dona Wong, a former graphics editor at the Wall Street Journal and Tufte's student, gives concise advice on colors, chart types, and clarity. This book is my first recommendation for anyone interested in data visualization.

The Big Book of Dashboards: Visualizing Your Data Using Real-World Business Scenarios

The big book of dashboards provides a great introduction to key data visualization principles and then presents valuable examples of dashboards that you may see in real life.

Fundamentals of Data Visualization: A Primer on Making Informative and Compelling Figures

In the fundamentals of data visualization, Wilke shows countless examples that differentiate a good chart from a bad chart. This book is special because Wike created his own plotting style available via an R package.

Observe Collect Draw!: A Visual Journal

Observe, collect, draw is a special book. One of my design professors recommended it to me. While it is not strictly a data visualization book of Tufte or Few variety, it provides great examples of how even simple data can be viewed with the help of a graphic. This book is great for ideation.

How Charts Lie: Getting Smarter about Visual Information

Alberto Cairo offers many examples of how the chart creators knowingly or unknowingly "lie." He shows how they do this: using faulty axis starting points, aspect ratios, and outright wrong data. This book will help you see how charts can manipulate certain emotions, and you will learn to question or avoid such charts.

Data Science for Fundraising: Build Data-Driven Solutions Using R

Yes, I co-wrote this book. Yes, I am biased. The data visualization chapter in this book is so detailed and long that it could have become a separate book. I wrote about principles of effective data visualization, summarizing my learnings from the books before me. I advised which charts to use. And I provided detailed R code to create your own effective graphics.

Sale

Visualizing Data

• Very nice appearance
• Hardcover Book
• Cleveland, William S. (Author)
• English (Publication Language)
• 360 Pages - 06/21/1993 (Publication Date) - Hobart Pr (Publisher)

$69.00

Buy on Amazon

Sale

The Visual Display of Quantitative Information, 2nd Ed.

• Hardcover Book
• Edward R. Tufte (Author)
• English (Publication Language)
• 197 Pages - 01/14/1997 (Publication Date) - Graphics Pr (Publisher)

$22.54

Buy on Amazon

Sale

Show Me the Numbers: Designing Tables and Graphs to Enlighten

• Used Book in Good Condition
• Hardcover Book
• Few, Stephen (Author)
• English (Publication Language)
• 371 Pages - 06/01/2012 (Publication Date) - Analytics Press (Publisher)

$20.24

Buy on Amazon

Sale

Storytelling with Data: A Data Visualization Guide for Business Professionals

• Wiley
• Language: english
• Book - storytelling with data: a data visualization guide for business professionals
• Nussbaumer Knaflic, Cole (Author)
• English (Publication Language)

$26.99

Buy on Amazon

Sale

Good Charts: The HBR Guide to Making Smarter, More Persuasive Data Visualizations

• Berinato, Scott (Author)
• English (Publication Language)
• 264 Pages - 05/17/2016 (Publication Date) - Harvard Business Review Press (Publisher)

$30.21

Buy on Amazon

Sale

The Wall Street Journal Guide to Information Graphics: The Dos and Don'ts of Presenting Data,...

• Wong, Dona M. (Author)
• English (Publication Language)
• 160 Pages - 12/16/2013 (Publication Date) - W. W. Norton & Company (Publisher)

$19.44

Buy on Amazon

Sale

The Big Book of Dashboards: Visualizing Your Data Using Real-World Business Scenarios

• Wexler, Steve (Author)
• English (Publication Language)
• 448 Pages - 04/24/2017 (Publication Date) - Wiley (Publisher)

$22.91

Buy on Amazon

Sale

Fundamentals of Data Visualization: A Primer on Making Informative and Compelling Figures

• Wilke, Claus O. (Author)
• English (Publication Language)
• 387 Pages - 05/14/2019 (Publication Date) - O'Reilly Media (Publisher)

$49.29

Buy on Amazon

Sale

Observe Collect Draw!: A Visual Journal

• Lupi, Giorgia (Author)
• English (Publication Language)
• 160 Pages - 09/25/2018 (Publication Date) - Princeton Architectural Press (Publisher)

$18.93

Buy on Amazon

Sale

How Charts Lie: Getting Smarter about Visual Information

• Hardcover Book
• Cairo, Alberto (Author)
• English (Publication Language)
• 256 Pages - 10/15/2019 (Publication Date) - W. W. Norton & Company (Publisher)

$19.95

Buy on Amazon

Data Science for Fundraising: Build Data-Driven Solutions Using R

• Nandeshwar, Ashutosh R (Author)
• English (Publication Language)
• 568 Pages - 02/14/2018 (Publication Date) - Data Insight Partners LLC (Publisher)

$53.99

Buy on Amazon

The post Best Books on Data Visualization appeared first on nandeshwar.info.

The argument of pie charts vs. bar charts is almost 100 years old, going back to Walter Eells’ paper titled “The Relative Merits of Circles and Bars for Representing Component Parts.” While pie charts are more common in business presentations, bar charts are finding increased use. There are advantages and disadvantages to both. In this post, you will learn about those as well as see alternatives.

Pie Charts

When to use a pie chart?

Analysts create pie charts to show the distribution of proportions, such as the percent of votes different candidates get in an election, or the proportion of revenue streams for a company. This is the simplest way to show “parts of the whole.”

Figure 1: Simple pie chart

Pie charts become confusing or unclear when many proportions, hence slices, are present. They are especially challenging when the slices are small. To solve this problem, creators use colors and legends to differentiate slices, but end up worsening the problem.

For example, compare charts I, II, and III below.

Figure 2: Three different pie charts with colors and legends

In the chart I, it is hard to differentiate between G and F. Chart II solves the differentiation problem, but we don’t know the value of G or F. Chart III tries to help with a legend, which may be needed if the labels are long, but cause another problem: looking for the key every time.

A potential solution is adding direct labels with data to each slice. While labels help clarify and offer precision, they defeat the purpose of creating a chart.

Figure 3: Pie charts with data labels

We can create better pie charts by reducing the number of slices by collapsing some of them or highlighting the important ones.

Let’s see these one at a time.

Reduced number of slices

We will put all the small ones in the “other” category.

Figure 4: Pie chart with collapsed categories

Highlight an important slice

We will keep the color of all slices the same, except for the one we want to highlight. Highlighting can be used to make a point along with an annotation.

Figure 5: Pie chart with a slice highlighted

Advantages of a pie chart

• You can see the portion of a category as compared to other the portions
• It offers an easy way to compare large and small slices
• It is intuitive

Disadvantages of a pie chart

• It makes comparison harder of almost similar sized slices
• With too many proportions or slices, it is harder to see the difference
• When colors or legends are used to differentiate slices, understanding is harder
• Too many labels crowd the chart, and the main point is lost

Bar charts

When to use bar charts?

Analysts create bar charts for a variety of uses: to show the absolute or proportional value i.e. total sales by product or proportion of a product sales of the total sales. Sometimes analysts create bar charts to show year over year numbers.

Although bar charts don’t show a part of the whole intuitively, they make comparisons of each proportion easy.

See the example below. You can see that the categories C and D are of the same size, and category G is the smallest.

Figure 6: Simple bar chart

This chart can be made better by arranging (or sorting) the bars by their lengths.

Figure 7: Ordered bar chart

We can make this chart aesthetically pleasing by placing invisible or white gridlines.

Figure 8: Ordered bar chart with white gridlines

Bar charts (the vertical ones) also have problems when the labels are long. Often they are at an angle, and you need to turn your head to read them.

Figure 9: Bar charts with long labels

You can solve this problem by moving the labels to the y-axis:

Figure 10: Bar charts with long labels flipped on the y-axis

Although the bar charts make the comparison easier than a pie chart, knowing the size of the smaller bars is similarly hard as knowing the size of a small slice from a pie chart. But at least the bar charts have gridlines to help us.

Just like the pie charts, we can try to overcome this problem by adding labels to the bars.

Figure 11: Bar charts with long labels flipped on the y-axis with some data labels

Advantages of bar charts

• Comparison among categories is easier
• Gridlines help guide the reader
• Length of the bar makes understanding easier
• Color coding and legends are not required (for single bars)
• Direct labeling is easier

Disadvantages of bar charts

• Smaller bars don’t offer precision
• Hard to “see” the proportion, that is the part of the whole

When to use a pie chart vs. bar chart

Use pie charts when:

• The number of categories is small
• Readers can differentiate slices (unless you are making a point)
• You don’t need to rely on many colors or labels to explain the proportions
• The total adds up to 100%

Use bar charts when:

• You have many categories (not too many)
• You need to compare numbers side-by-side (caution: more than two bars are hard for readers

Waffle Charts

Waffle charts combine bar charts and pie charts into one chart and compound their problems. While you can see the proportions, precision is lost, especially when the proportions don’t fit in one square. And, you still have to decode the color and legend to understand the chart. Also, longer labels will create problems with legend positioning. These charts could be useful with fewer categories, or if analysts want to make a point.

Figure 12: A waffle chart

Dot charts

Dot charts or dot plots, popularized by William Cleveland, are similar to bar plots, except the bars are replaced with a dot at the end of the bar.

It’s easier to see an example:

Figure 13: A simple dot chart

Dot charts use the best qualities of a bar chart such as ordered data, labels on the y-axis, and reducing the need to rely on color for decoding the data. It exceeds the usefulness of a bar chart by using a smaller space than a bar chart.

Cleveland said this about area and perception, “Using area to encode quantitative information is a poor graphical method. Effects that can be readily perceived in other visualizations are often lost in an encoding by area.”

Advantages of dot charts

• Comparison is easier
• Take little space
• Differences among categories are noticeable
• Color or legend coding not required (for single categories)

Disadvantages of dot charts

• Some precision is lost
• They might be new to viewers who may find the charts dull

Head to head: pie charts vs. bar charts

Let’s look at how pie charts compare against bar charts with real data. These data are from: “Reveal from The Center for Investigative Reporting and The Center for Employment Equity,” a study on diversity in silicon valley.

First up, let’s look at all the distribution of IT professionals by roles.

Figure 14: Bar chart vs. pie chart comparing job categories of employees in Silicon Valley companies

Figure 15: Bar chart vs. pie chart comparing job categories of employees in Silicon Valley companies with reduced height

What do we see

• Both the pie and bar chart in Figure 14 clearly show that professionals account for more than 50% of the workforce.
• It is hard to see the percentage of executives.
• Both the charts clearly show that there more “other workers” than there are managers.
• It is hard to see the percentage of “other workers” in the pie chart, but you can see that it is slightly below 30% in the bar chart.
• When we reduced the height of both plots by 50% in Figure 15, you can see that the bar chart is still readable and gives you the same information as Figure 14

Verdict: slight advantage bar charts. If we add data labels to both the charts, they will be at the same level.

Now, let’s look at the distribution of ethnicities/races.

Here’s another head to head: pie charts vs. bar charts round II.

Figure 15: Bar chart vs. pie chart comparing ethnicities/races of employees in Silicon Valley companies

What do we see

• Both the pie and bar chart in Figure 15 clearly show that white employees account for more than 50% of the workforce.
• Both the charts show that Asian employees account for slightly over 25% of the workforce
• The bar chart shows that there are almost double the number of Latinx employees than Black employees. We can see the same ratio for Asian and white employees in the bar chart.
• It is hard to see the percentage of “other” and Black employees.

Verdict: slight advantage bar charts. With data labels, the pie chart may come out ahead.

What if we want to see the job category and race/ethnicity at the same time. We can use stack bar charts or side-by-side bar charts, but what about the pie charts?

We can combine the ethnicity/race with the job category and create more slices. We can also use hues for different ethnicity/races and place them close to each other. Round III of pie charts vs. bar charts .

Job category and race/ethnicity

Figure 16: A pie chart showing ethnicities/races and job categories of employees in Silicon Valley companies

Figure 17: A bar chart showing ethnicities/races and job categories of employees in Silicon Valley companies

What do we see

• Many labels in the pie chart (Figure 16) are unreadable
• We see white “professionals” account for most of the IT workforce, followed by Asian “professionals”. This can be seen clearly in both the charts (Figure 16 and 17)
• In the bar chart (Figure 17), you can see managers who are white makeup for the most number of managers.
• In the bar chart, you can see the non-existence of Latinx, Black, and other executives though the data does contain a small percentage of executives from these groups.
• Although you can’t say it with precision, in the bar chart, you can see the ratio of Asian managers to Asian professionals compared to white managers to white professionals is smaller. [It is 0.2 compared to .37]

Verdict: meh! Except for one or two large patterns, you have to search into the charts to make meaningful observations.

We can try facet or “small multiples” to see whether these charts are any better. Let’s see panels of ethnicity/race and job category distribution. Round IV of panel pie charts vs. bar charts.

Small multiples

Panels of ethnicity/race with job category distribution

Figure 18: Pie chart panels showing job categories of employees in Silicon Valley companies by ethnicity/race

Figure 19: Bar chart panels showing job categories of employees in Silicon Valley companies by ethnicity/race

What do we see

• You have to jump from one chart to another to make comparisons, more so with pie charts
• In the pie charts (Figure 18), you see:
  • A large percentage of Asian employees are “professionals”
  • More than half of the Black and Latinx employees fall in the “Other workers” category
• In the bar charts (Figure 19), you see:
  • The ratio of managers to “professionals” look smaller compared to the other ethnicities/races
  • The ratio of “other workers” to “professionals” look bigger for Black and Latinx employees compared to the other ethnicities/races

Verdict: tie. Both charts offer a different understanding. The pie charts show underrepresentation or over-representation more intuitively, but the comparison between graphs is harder than the bar charts.

Panels of job category by ethnicity/race distribution

Let’s see panels of job category by ethnicity/race distribution. Round V of pie charts vs. bar charts.

Figure 20: Pie chart panels showing ethnicity/race of employees in Silicon Valley companies by job category

Figure 21: Bar chart panels showing ethnicity/race of employees in Silicon Valley companies by job category

What do we see

• In the pie charts (Figure 20), you can see:
  • white employees account for a large portion of the executives
  • Asian employees have a large percentage of “professionals”
  • Latinx employees have a higher representation in the “other workers” categories compared to the other job categories.
• In the bar charts (Figure 21), you can see:
  • Black employees are underrepresented in all of the categories, except for the “other workers” category.

Verdict: pie charts are slightly ahead. Maybe because the 73% and 52% of white employees in the executive and “professionals” categories respectively jumped out to me. The data labels are harder to see for smaller slices, and if we added those to the bar charts, it could be a tie.

Dot charts

Figure 22: A dot chart of race/ethnicity distribution within job category

Figure 23: A dot chart of job category distribution within race/ethnicity

What do we see

• In the dot chart of Figure 22, in which we compare ethnicity/race within the job categories, we see:
  • since we have more width available compared to the bar charts, the gap between white executives (72%) and Asian executives (21%) looks larger than shown in the bar charts
  • The comparison among groups is easier within the panel and with other panels. You don’t need to read the y-axis labels every time.
  • You can see the wide gap between white/Asian “professionals” and Latinx/Black/Other “professionals.”
• In the dot chart of Figure 23, in which we compare job categories within the ethnicities/races, we see:
  • there are many more Asian “professionals” than there are executives or managers
  • there are many more Black/Latinx “other workers” than there are “professionals,” executives, or managers

The dot charts offer more information in a small space compared to the bar or pie charts. The comparison is easier though some precision is lost. You don’t have to use color to distinguish categories similar to bar charts. They also look very clean.

What if we want to compare genders within the job categories and ethnicities/races?

I doubt we will get good, easily explainable graphs. Simple bar charts and pie charts are out. We can try dot charts with two dots for the genders in the data, and side-by-side bar charts.

Here are three attempts:

• Figure 24 shows gender representation within a job category by race. When you add the values of all purple bars or green bars in a job category, you will get a total of 100%.
• Figure 25 is same as Figure 24, but in a dot chart form.
• Figure 26 shows the distribution of all genders and ethnicities/races within a job category. When you add all the values within a job category, you will get a total of 100%.

While Figure 24 and 25 show genders compare among ethnicities/races within a job category, Figure 26 shows how genders from all ethnicities/races compare with each other within a job category.

Figure 24: Side-by-side bar charts of job categories and ethnicity/race distribution by gender

Figure 25: Dot charts of job categories and ethnicity/race distribution by gender

Figure 26: Dot charts of job categories and ethnicity/race distribution by gender

A note on dot charts

As we saw in the previous examples, pie charts aren’t suitable for multi-level comparison, and although bar charts are good alternatives, dot charts offer more flexibility conveying similar information without a loss of perception or understanding. Dot charts don’t require color-coding also because we can use different symbols (patterned bar charts look ugly. I know because I have created many of them before).

Conclusion

I was surprised by some of the graphs. Pie charts were better in some instances, and side-by-side bar charts were better than dot charts in at least one case. That is why you need to create multiple designs before settling on one. And of course, it also depends on the objective of your overall narrative. For example, Figure 24 and 26 have similar data, but they show two different things. Context is critical. Also, important are design skills – some graphs out of the box may not be ready for sharing, but with editing and annotating, charts can speak for themselves.

Appendix: R Code

####################################################################################################
## ----setup


knitr::opts_chunk$set(
  echo = FALSE, message = FALSE, warning = FALSE,
  fig.width = 6,
  fig.align = "center",
  dpi = 96
)

library(scales)
library(ggplot2)
library(ggforce)
library(patchwork)
library(dplyr)
library(ggthemes)
library(waffle)
library(readr)
library(RColorBrewer)
library(ggtext)

race_colors <- c("White" = "#9e9ac8", "Asian" = "#6baed6", "Latinx" = "#fd8d3c", "Black" = "#74c476", "Other" = "#fb6a4a")
job_cat_colors <- c("Other workers" = "#8da0cb", "Professionals" = "#e78ac3", "Managers" = "#fc8d62", "Executives" = "#66c2a5")

ordered_lvls_race <- c("White", "Asian", "Latinx", "Black", "Other")
ordered_lvls_job_cat <- c("Executives", "Managers", "Professionals", "Other workers")
####################################################################################################



####################################################################################################
## Simple pie chart in R
par(
  mar = c(rep(.8, 4)),
  mai = rep(0.1, 4)
)
pie(c(0.3, 0.4, 0.3), labels = c("A", "B", "C"), col = "grey80", border = "white")
####################################################################################################

####################################################################################################
## Three different pie charts with colors and legends
data <- data.frame(
  val = c(0.3, 0.4, 0.1, 0.1, 0.05, 0.02, 0.01, 0.02),
  cat = LETTERS[1:8],
  long_cat = c("Cream of Wheat", "Malt-O-Meal", "Maypo", "Quaker Oats", "Cinnamon Crunch", "Scott's Porage Oats", "Cap'n Crunch", "Cheerios"),
  stringsAsFactors = FALSE
)


# copied from Claus Wilke beautiful pie chart in r 
# https://github.com/clauswilke/dataviz/blob/master/nested_proportions.Rmd
# I haven't paid attention to know how this works
# create a pie chart from data frame in r

data <- data %>%
  arrange(val) %>%
  mutate(
    end_angle = 2 * pi * cumsum(val) / sum(val), 
    start_angle = lag(end_angle, default = 0), 
    mid_angle = 0.5 * (start_angle + end_angle), 
    hjust = ifelse(mid_angle > pi, 1, 0),
    vjust = ifelse(mid_angle < pi / 2 | mid_angle > 3 * pi / 2, 0, 1)
  )

rpie <- 1
rlabel_out <- 1.05 * rpie
rlabel_in <- 0.6 * rpie


p1 <- ggplot(data) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle
    ),
    fill = "grey90",
    color = "white"
  ) +
  coord_fixed() +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = cat,
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  ggtitle("I") +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  ) +
  theme(plot.title = element_text(color = "#1A3BA5", hjust = 0.5, face = "bold", size = rel(1.5)))

p2 <- ggplot(data) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle,
      fill = cat,
    ),
    color = "white"
  ) +
  coord_fixed() +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = cat,
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  ggtitle("II") +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  ) +
  scale_fill_brewer(type = "qual", palette = "Set2") +
  theme(legend.position = "none") +
  theme(plot.title = element_text(color = "#1A3BA5", hjust = 0.5, face = "bold", size = rel(1.5)))

p3 <- ggplot(data) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle,
      fill = cat,
    ),
    color = "gray85"
  ) +
  coord_fixed() +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = cat,
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  ggtitle("III") +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  ) +
  scale_fill_brewer(type = "seq", palette = "Oranges") +
  theme(plot.title = element_text(color = "#1A3BA5", hjust = 0.5, face = "bold", size = rel(1.5)))


p1 + p2 + p3
####################################################################################################


####################################################################################################
## Pie charts with data labels
p4 <- ggplot(data) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle
    ),
    fill = "grey90",
    color = "white"
  ) +
  coord_fixed(clip = "off") +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = paste(cat, percent(val), sep = ": "),
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  ggtitle("Label outside") +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  ) +
  theme(plot.title = element_text(color = "#1A3BA5", hjust = 0.5, face = "bold", size = rel(1.5)))


p5 <- ggplot(data) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle
    ),
    fill = "grey90",
    color = "white"
  ) +
  coord_fixed(clip = "off") +
  geom_text(
    aes(
      x = rlabel_in * sin(mid_angle),
      y = rlabel_in * cos(mid_angle),
      label = ifelse(val < .1, NA, paste(cat, percent(val), sep = ": "))
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  ggtitle("Label inside") +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  ) +
  theme(plot.title = element_text(color = "#1A3BA5", hjust = 0.5, face = "bold", size = rel(1.5)))


wrap_plots(p4, p5, heights = 1)
####################################################################################################

####################################################################################################
## Pie chart with collapsed categories

collapsed_data <- data.frame(
  val = c(0.3, 0.4, 0.1, 0.2),
  cat = c(LETTERS[1:3], "Other"),
  stringsAsFactors = FALSE
)

# copied from Claus Wilke https://github.com/clauswilke/dataviz/blob/master/nested_proportions.Rmd
# I haven't paid attention to know how this works

collapsed_data <- collapsed_data %>%
  arrange(val) %>%
  mutate(
    end_angle = 2 * pi * cumsum(val) / sum(val), 
    start_angle = lag(end_angle, default = 0), 
    mid_angle = 0.5 * (start_angle + end_angle), 
    hjust = ifelse(mid_angle > pi, 1, 0),
    vjust = ifelse(mid_angle < pi / 2 | mid_angle > 3 * pi / 2, 0, 1)
  )

rpie <- 1
rlabel_out <- 1.05 * rpie
rlabel_in <- 0.6 * rpie


p6 <- ggplot(collapsed_data) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle
    ),
    fill = "grey90",
    color = "white"
  ) +
  coord_fixed(clip = "off") +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = paste(cat, percent(val), sep = ": "),
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  )

p6
####################################################################################################

####################################################################################################
## Pie chart with a slice highlighted

slice_colors <- rep("grey90", 8)
names(slice_colors) <- LETTERS[1:8]
slice_colors["F"] <- "#F3790C"

p7 <- ggplot(data) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle,
      fill = cat,
    ),
    color = "white"
  ) +
  coord_fixed(clip = "off") +
  scale_fill_manual(values = slice_colors) +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = cat,
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  ) +
  theme(legend.position = "none")

p7 <- p7 + geom_curve(aes(x = 0.15, y = 1.14, xend = .6, yend = 1), curvature = -0.6) +
  geom_text(aes(x = 0.61, y = 1, label = "25% growth likely\nnext year"),
    hjust = 0,
    color = "#F3790C"
  )

p7
####################################################################################################


####################################################################################################
## Simple bar chart
b1 <- ggplot(data, aes(x = cat, y = val)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_y_continuous(labels = percent) +
  theme_wsj() +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank()
  )

b1
####################################################################################################


####################################################################################################
## Ordered bar chart
b2 <- ggplot(data, aes(x = reorder(cat, desc(val)), y = val)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_y_continuous(labels = percent) +
  theme_wsj() +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank()
  )

b2
####################################################################################################


####################################################################################################
## Ordered bar chart with white gridlines
b3 <- ggplot(data, aes(x = reorder(cat, desc(val)), y = val)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_y_continuous(labels = percent) +
  theme_wsj() +
  geom_hline(yintercept = 1:3 / 10, color = "white") +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank(),
    panel.grid.major.y = element_blank()
  )

b3
####################################################################################################


####################################################################################################
## Bar charts with long labels
b4 <- ggplot(data, aes(x = reorder(long_cat, desc(val)), y = val)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_y_continuous(labels = percent) +
  theme_wsj() +
  geom_hline(yintercept = 1:3 / 10, color = "white") +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank(),
    panel.grid.major.y = element_blank()
  )

b4_i <- b4 + theme(axis.text.x = element_text(angle = 90))

b4_ii <- b4 + theme(axis.text.x = element_text(angle = 30, hjust = 1))

wrap_plots(b4_i, b4_ii, heights = 1)
####################################################################################################


####################################################################################################
## Bar charts with long labels flipped on the y-axis
b5 <- ggplot(data, aes(y = reorder(long_cat, val), x = val)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_x_continuous(labels = percent) +
  theme_wsj() +
  geom_vline(xintercept = 1:3 / 10, color = "white") +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank(),
    panel.grid.major.y = element_blank()
  )

b5
####################################################################################################


####################################################################################################
## Bar charts with long labels flipped on the y-axis with some data labels
b6 <- ggplot(data, aes(y = reorder(long_cat, val), x = val)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_x_continuous(labels = percent) +
  theme_wsj() +
  geom_vline(xintercept = 1:3 / 10, color = "white") +
  geom_text(aes(label = ifelse(val < 0.1, percent(val), "")), hjust = -0.1) +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank(),
    panel.grid.major.y = element_blank()
  )

b6
####################################################################################################


####################################################################################################
## A waffle chart
data2 <- arrange(data, val)
parts_v <- data2$val * 100
names(parts_v) <- data2$cat

w1 <- waffle(parts = parts_v, rows = 5, legend_pos = "top", xlab = "1 square equals 1%", reverse = TRUE)

w1 <- w1 + guides(fill = guide_legend(
  nrow = 1,
  reverse = TRUE,
  label.position = "top"
)) +
  theme(legend.spacing.x = unit(1.2, "cm"))

w1
####################################################################################################


####################################################################################################
## A simple dot chart
d1 <- ggplot(data, aes(y = reorder(cat, val), x = val)) +
  geom_point(shape = 21, fill = "#F3790C", size = 3) +
  scale_x_continuous(labels = percent) +
  theme_wsj() +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank(),
    panel.grid.major.y = element_line(size = 0.4)
  )

d1
####################################################################################################

# IT Diversity Silicon Valley  data
# Is Silicon Valley Tech Diversity Possible Now?
# Center for Employment Equity, University of Massachuset
# https://github.com/cirlabs/Silicon-Valley-Diversity-Data
# https://www.umass.edu/employmentequity/sites/default/files/CEE_Diversity%2Bin%2BSilicon%2BValley%2BTech.pdf

it_data_clean <- read_csv("tech_diversity_cleaned.csv")


####################################################################################################
## Bar chart vs. pie chart comparing job categories of employees in Silicon Valley companies
job_cat_total <- it_data_clean %>%
  group_by(job_category) %>%
  summarize(total = sum(count))


job_cat_total <- job_cat_total %>%
  arrange(total) %>%
  mutate(
    end_angle = 2 * pi * cumsum(total) / sum(total), 
    start_angle = lag(end_angle, default = 0),
    mid_angle = 0.5 * (start_angle + end_angle), 
    hjust = ifelse(mid_angle > pi, 1, 0),
    vjust = ifelse(mid_angle < pi / 2 | mid_angle > 3 * pi / 2, 0, 1)
  ) %>%
  mutate(pct = total / sum(total))

rpie <- 1
rlabel_out <- 1.05 * rpie
rlabel_in <- 0.6 * rpie


p1_it <- ggplot(job_cat_total) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle
    ),
    fill = "grey90",
    color = "white"
  ) +
  coord_fixed(clip = "off") +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = job_category,
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  )




b1_it <- ggplot(job_cat_total, aes(y = reorder(job_category, pct), x = pct)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_x_continuous(labels = percent, limits = c(0, .6)) +
  theme_wsj() +
  geom_vline(xintercept = 1:6 / 10, color = "white") +
  # geom_text(aes(label = ifelse(val < 0.1, percent(val), "")), hjust = -0.1) +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank(),
    panel.grid.major.y = element_blank(),
    plot.title = element_text(family = "", size = 2)
  )


wrap_plots(p1_it, b1_it, heights = 1) +
  plot_annotation(
    caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/",
    theme = theme(
      plot.caption = element_text(family = "", size = 9, hjust = 0),
      plot.caption.position = "plot"
    )
  )
####################################################################################################


####################################################################################################
## Bar chart vs. pie chart comparing ethnicities/races of employees in Silicon Valley companies
race_total <- it_data_clean %>%
  group_by(race_short) %>%
  summarize(total = sum(count)) %>%
  ungroup()


race_total <- race_total %>%
  arrange(total) %>%
  mutate(
    end_angle = 2 * pi * cumsum(total) / sum(total), 
    start_angle = lag(end_angle, default = 0), 
    mid_angle = 0.5 * (start_angle + end_angle), 
    hjust = ifelse(mid_angle > pi, 1, 0),
    vjust = ifelse(mid_angle < pi / 2 | mid_angle > 3 * pi / 2, 0, 1)
  ) %>%
  mutate(pct = total / sum(total))

rpie <- 1
rlabel_out <- 1.05 * rpie
rlabel_in <- 0.6 * rpie


p2_it <- ggplot(race_total) +
  geom_arc_bar(
    aes(
      x0 = 0, y0 = 0, r0 = 0, r = rpie,
      start = start_angle, end = end_angle
    ),
    fill = "grey90",
    color = "white"
  ) +
  coord_fixed(clip = "off") +
  geom_text(
    aes(
      x = rlabel_out * sin(mid_angle),
      y = rlabel_out * cos(mid_angle),
      label = race_short,
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  theme_void() +
  scale_x_continuous(
    name = NULL,
    limits = c(-1.5, 1.4),
    expand = c(0, 0)
  ) +
  scale_y_continuous(
    name = NULL,
    limits = c(-1.2, 1.3),
    expand = c(0, 0)
  )




b2_it <- ggplot(race_total, aes(y = reorder(race_short, pct), x = pct)) +
  geom_bar(stat = "identity", fill = "gray80") +
  scale_x_continuous(labels = percent, limits = c(0, .6)) +
  theme_wsj() +
  geom_vline(xintercept = 1:6 / 10, color = "white") +
  # geom_text(aes(label = ifelse(val < 0.1, percent(val), "")), hjust = -0.1) +
  theme(
    panel.background = element_blank(),
    plot.background = element_blank(),
    axis.title = element_blank(),
    panel.grid.major.y = element_blank(),
    plot.title = element_text(family = "", size = 2)
  )

wrap_plots(p2_it, b2_it, heights = 1) +
  plot_annotation(
    caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/",
    theme = theme(
      plot.caption = element_text(family = "", size = 9, hjust = 0),
      plot.caption.position = "plot"
    )
  )
####################################################################################################



####################################################################################################
## A pie chart showing ethnicities/races and job categories of employees in Silicon Valley companies
race_job_cat_total <- it_data_clean %>%
  group_by(race_short, job_category) %>%
  summarize(total = sum(count)) %>%
  ungroup() %>%
  mutate(race_job_cat = paste(race_short, job_category, sep = "-")) %>%
  mutate(pct = total / sum(total))


race_job_cat_total <- race_job_cat_total %>%
  arrange(race_short, total) %>%
  mutate(
    count_total = sum(total),
    end_angle = 2 * pi * cumsum(total) / count_total, # ending angle for each pie slice
    start_angle = lag(end_angle, default = 0), # starting angle for each pie slice
    mid_angle = 0.5 * (start_angle + end_angle), # middle of each pie slice, for the text label
    hjust = ifelse(mid_angle > pi, 1, 0),
    vjust = ifelse(mid_angle < pi / 2 | mid_angle > 3 * pi / 2, 0, 1),
    type = job_category,
    label = race_job_cat
  )

slice_colors <- c(
  brewer.pal(5, "Blues")[-1],
  brewer.pal(5, "Greens")[-1],
  brewer.pal(5, "Oranges")[-1],
  brewer.pal(5, "Reds")[-1],
  brewer.pal(5, "Purples")[-1]
)

names(slice_colors) <- race_job_cat_total$race_job_cat

rpie <- 1
rpie1 <- 0
rpie2 <- 1
rlabel <- 1.02 * rpie

job_cat_race_nested_pie <- ggplot() +
  geom_arc_bar(
    data = race_job_cat_total,
    aes(
      x0 = 0, y0 = 0, r0 = rpie1, r = rpie2,
      start = start_angle, end = end_angle, fill = race_job_cat
    ),
    color = "white", size = 0.5
  ) +
  geom_text(
    data = race_job_cat_total,
    aes(
      x = rlabel * sin(mid_angle),
      y = rlabel * cos(mid_angle),
      label = race_job_cat,
      hjust = hjust, vjust = vjust
    ),
    size = 12 / .pt
  ) +
  geom_text(
    data = race_job_cat_total,
    aes(
      x = 0.6 * sin(mid_angle),
      y = 0.6 * cos(mid_angle),
      label = percent(pct, accuracy = 1)
    ),
    size = 10 / .pt,
    hjust = 0.5, vjust = 0.5
  ) +
  coord_fixed(clip = "off") +
  scale_x_continuous(
    limits = c(-1.5, 1.8), expand = c(0, 0), name = "", breaks = NULL, labels = NULL
  ) +
  scale_y_continuous(
    limits = c(-1.15, 1.15), expand = c(0, 0), name = "", breaks = NULL, labels = NULL
  ) +
  scale_fill_manual(
    values = slice_colors
  ) +
  labs(
    title = "Colorful, bad design",
    caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/"
  ) +
  theme(
    legend.position = "none",
    panel.background = element_blank(),
    plot.background = element_blank(),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot",
    plot.title.position = "plot",
    plot.title = element_text(face = "bold", size = 12)
  )

job_cat_race_nested_pie
####################################################################################################


####################################################################################################
## A bar chart showing ethnicities/races and job categories of employees in Silicon Valley companies

ordered_lvls_job_cat_race_combined <- c("White-Executives", "White-Managers", "White-Professionals", "White-Other workers", "Asian-Executives", "Asian-Managers", "Asian-Professionals", "Asian-Other workers", "Latinx-Executives", "Latinx-Managers", "Latinx-Professionals", "Latinx-Other workers", "Black-Executives", "Black-Managers", "Black-Professionals", "Black-Other workers", "Other-Executives", "Other-Managers", "Other-Professionals", "Other-Other workers")

job_cat_race_nested_bar <- ggplot(race_job_cat_total, aes(
  y = factor(race_job_cat, levels = rev(ordered_lvls_job_cat_race_combined)),
  x = pct,
  group = race_short,
  fill = job_category
)) +
  geom_bar(stat = "identity", position = position_dodge(0.5)) +
  scale_fill_brewer(type = "qual", palette = "Set2") +
  scale_x_continuous(labels = percent) +
  scale_y_discrete(expand = c(0, 0)) +
  geom_vline(xintercept = seq(from = 0, to = .3, by = 0.05), color = "white") +
  theme_minimal() +
  labs(
    title = "Colorful, bad design",
    caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/"
  ) +
  theme(
    legend.position = "top",
    legend.title = element_blank(),
    panel.grid = element_blank(),
    axis.text = element_text(face = "bold", size = 12),
    axis.title = element_blank(),
    axis.text.y = element_text(hjust = 1),
    axis.text.x = element_text(hjust = 0.2),
    axis.line = element_line(),
    axis.line.y = element_blank(),
    axis.ticks.y = element_blank(),
    axis.ticks.x = element_line(colour = NULL),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot",
    plot.title.position = "plot",
    plot.title = element_text(face = "bold", size = 12)
  ) +
  guides(fill = guide_legend(nrow = 1))

job_cat_race_nested_bar
####################################################################################################


####################################################################################################
## Pie chart panels showing job categories of employees in Silicon Valley companies by ethnicity/race

race_job_cat_total <- it_data_clean %>%
  group_by(race_short, job_category) %>%
  summarize(total = sum(count)) %>%
  mutate(pct = total / sum(total))


race_job_cat_total <- race_job_cat_total %>%
  arrange(race_short, total) %>%
  group_by(race_short) %>%
  mutate(
    count_total = sum(total),
    end_angle = 2 * pi * cumsum(total) / count_total, 
    start_angle = lag(end_angle, default = 0), 
    mid_angle = 0.5 * (start_angle + end_angle), 
    hjust = ifelse(mid_angle > pi, 1, 0),
    vjust = ifelse(mid_angle < pi / 2 | mid_angle > 3 * pi / 2, 0, 1),
    type = job_category,
    label = job_category
  )

race_job_cat_total <- ungroup(race_job_cat_total) %>%
  mutate(race_short = factor(race_short, ordered_lvls_race))



rpie <- 1
rpie1 <- 0
rpie2 <- 1
rlabel <- 1.02 * rpie

job_cat_race_panel_pie_v1 <- ggplot() +
  geom_arc_bar(
    data = race_job_cat_total,
    aes(
      x0 = 0, y0 = 0, r0 = rpie1, r = rpie2,
      start = start_angle, end = end_angle, fill = job_category
    ),
    color = "white", size = 0.5
  ) +
  facet_wrap(~race_short) +
  geom_text(
    data = race_job_cat_total,
    aes(
      x = rlabel * sin(mid_angle),
      y = rlabel * cos(mid_angle),
      label = ifelse(job_category == "Other workers", "Other\nworkers", job_category),
      hjust = hjust, vjust = vjust
    ),
    # family = dviz_font_family,
    size = 10 / .pt
  ) +
  geom_text(
    data = race_job_cat_total,
    aes(
      x = 0.6 * sin(mid_angle),
      y = 0.6 * cos(mid_angle),
      label = percent(pct, accuracy = 1)
    ),
    size = 10 / .pt,
    hjust = 0.5, vjust = 0.5
  ) +
  coord_fixed(clip = "off") +
  scale_x_continuous(
    limits = c(-1.5, 1.8), expand = c(0, 0), name = "", breaks = NULL, labels = NULL
  ) +
  scale_y_continuous(
    limits = c(-1.15, 1.15), expand = c(0, 0), name = "", breaks = NULL, labels = NULL
  ) +
  scale_fill_brewer(type = "qual", palette = "Set2") +
  theme(
    legend.position = "none",
    panel.background = element_blank(),
    plot.background = element_blank()
  ) +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/")

job_cat_race_panel_pie_v1 <- job_cat_race_panel_pie_v1 +
  theme(
    strip.text = element_text(face = "bold", size = 12),
    strip.background = element_rect(fill = "grey95"),
    panel.spacing = unit(1.5, "cm"),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot"
  )

job_cat_race_panel_pie_v1
####################################################################################################


####################################################################################################
## Bar chart panels showing job categories of employees in Silicon Valley companies by ethnicity/race

race_job_cat_total <- it_data_clean %>%
  group_by(race_short, job_category) %>%
  summarize(total = sum(count)) %>%
  mutate(pct = total / sum(total))

race_job_cat_total <- ungroup(race_job_cat_total) %>%
  mutate(
    race_short = factor(race_short, ordered_lvls_race),
    job_category = factor(job_category, rev(ordered_lvls_job_cat))
  )



job_cat_race_panel_bar_v1 <- ggplot(race_job_cat_total, aes(
  y = job_category,
  x = pct,
  fill = job_category
)) +
  geom_bar(stat = "identity", position = position_dodge(0.5), width = 0.7) +
  facet_wrap(~race_short) +
  scale_fill_brewer(type = "qual", palette = "Set2") +
  scale_x_continuous(labels = percent) +
  geom_vline(xintercept = seq(from = 0, to = .6, by = 0.2), color = "white") +
  theme_minimal() +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/") +
  theme(
    legend.position = "none",
    legend.title = element_blank(),
    panel.grid = element_blank(),
    axis.text = element_text(face = "bold", size = 12),
    axis.title = element_blank(),
    axis.text.y = element_text(hjust = 1),
    axis.text.x = element_text(hjust = 0.2),
    axis.line = element_line(),
    axis.line.y = element_blank(),
    axis.ticks.y = element_blank(),
    axis.ticks.x = element_line(colour = NULL)
  ) +
  guides(fill = guide_legend(nrow = 1)) +
  theme(
    strip.text = element_text(face = "bold", size = 12),
    strip.background = element_rect(fill = "grey95", color = NA),
    panel.spacing = unit(1, "cm"),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot"
  )

job_cat_race_panel_bar_v1
####################################################################################################



####################################################################################################
## Pie chart panels showing ethnicity/race of employees in Silicon Valley companies by job category
race_job_cat_total <- it_data_clean %>%
  group_by(job_category, race_short) %>%
  summarize(total = sum(count)) %>%
  mutate(pct = total / sum(total))


race_job_cat_total <- race_job_cat_total %>%
  arrange(job_category, total) %>%
  group_by(job_category) %>%
  mutate(
    count_total = sum(total),
    end_angle = 2 * pi * cumsum(total) / count_total, # ending angle for each pie slice
    start_angle = lag(end_angle, default = 0), # starting angle for each pie slice
    mid_angle = 0.5 * (start_angle + end_angle), # middle of each pie slice, for the text label
    hjust = ifelse(mid_angle > pi, 1, 0),
    vjust = ifelse(mid_angle < pi / 2 | mid_angle > 3 * pi / 2, 0, 1),
    type = job_category,
    label = job_category
  )

race_job_cat_total <- ungroup(race_job_cat_total) %>%
  mutate(
    race_short = factor(race_short, rev(ordered_lvls_race)),
    job_category = factor(job_category, ordered_lvls_job_cat)
  )

rpie <- 1
rpie1 <- 0
rpie2 <- 1
rlabel <- 1.02 * rpie


job_cat_race_panel_pie_v2 <- ggplot() +
  geom_arc_bar(
    data = race_job_cat_total,
    aes(
      x0 = 0, y0 = 0, r0 = rpie1, r = rpie2,
      start = start_angle, end = end_angle, fill = race_short
    ),
    color = "white", size = 0.5
  ) +
  facet_wrap(~job_category) +
  geom_text(
    data = race_job_cat_total,
    aes(
      x = rlabel * sin(mid_angle),
      y = rlabel * cos(mid_angle),
      label = race_short,
      hjust = hjust, vjust = vjust
    ),
    size = 10 / .pt
  ) +
  geom_text(
    data = race_job_cat_total,
    aes(
      x = 0.6 * sin(mid_angle),
      y = 0.6 * cos(mid_angle),
      label = percent(pct, accuracy = 1)
    ),
    size = 10 / .pt,
    hjust = 0.5, vjust = 0.5
  ) +
  coord_fixed(clip = "off") +
  scale_x_continuous(
    limits = c(-1.5, 1.8), expand = c(0, 0), name = "", breaks = NULL, labels = NULL
  ) +
  scale_y_continuous(
    limits = c(-1.15, 1.15), expand = c(0, 0), name = "", breaks = NULL, labels = NULL
  ) +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/") +
  scale_fill_manual(values = race_colors) +
  theme(
    legend.position = "none",
    panel.background = element_blank(),
    plot.background = element_blank()
  )

job_cat_race_panel_pie_v2 <- job_cat_race_panel_pie_v2 +
  theme(
    strip.text = element_text(face = "bold", size = 12),
    strip.background = element_rect(fill = "grey95"),
    panel.spacing = unit(1.5, "cm"),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot"
  )

job_cat_race_panel_pie_v2
####################################################################################################


####################################################################################################
## Bar chart panels showing ethnicity/race of employees in Silicon Valley companies by job category


race_job_cat_total <- ungroup(race_job_cat_total) %>%
  mutate(
    race_short = factor(race_short, rev(ordered_lvls_race)),
    job_category = factor(job_category, ordered_lvls_job_cat)
  )

job_cat_race_panel_bar_v2 <- ggplot(race_job_cat_total, aes(
  y = race_short,
  x = pct,
  fill = race_short
)) +
  geom_bar(stat = "identity", position = position_dodge(0.5), width = 0.7) +
  facet_wrap(~job_category) +
  scale_fill_manual(values = race_colors) +
  scale_x_continuous(labels = percent) +
  scale_y_discrete(expand = c(0, 0)) +
  geom_vline(xintercept = seq(from = 0, to = .6, by = 0.2), color = "white") +
  theme_minimal() +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/") +
  theme(
    legend.position = "none",
    legend.title = element_blank(),
    panel.grid = element_blank(),
    axis.text = element_text(face = "bold", size = 12),
    axis.title = element_blank(),
    axis.text.y = element_text(hjust = 1),
    axis.text.x = element_text(hjust = 0.2),
    axis.line = element_line(),
    axis.line.y = element_blank(),
    axis.ticks.y = element_blank(),
    axis.ticks.x = element_line(colour = NULL)
  ) +
  guides(fill = guide_legend(nrow = 1)) +
  theme(
    strip.text = element_text(face = "bold", size = 12),
    strip.background = element_rect(fill = "grey95", color = NA),
    panel.spacing = unit(1, "cm"),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot"
  )

job_cat_race_panel_bar_v2
####################################################################################################


####################################################################################################
## A dot chart of race/ethnicity distribution within job category

race_job_cat_distribution_within_jobcat <- it_data_clean %>%
  group_by(race_short, job_category) %>%
  summarize(total = sum(count)) %>%
  group_by(race_short) %>%
  mutate(pct = total / sum(total))

race_job_cat_distribution_within_race <- it_data_clean %>%
  group_by(race_short, job_category) %>%
  summarize(total = sum(count)) %>%
  group_by(job_category) %>%
  mutate(pct = total / sum(total))


race_job_cat_distribution_within_race <- ungroup(race_job_cat_distribution_within_race) %>%
  mutate(
    race_short = factor(race_short, rev(ordered_lvls_race)),
    job_category = factor(job_category, ordered_lvls_job_cat)
  )



race_within_job_cat_dot_plot <- ggplot(
  race_job_cat_distribution_within_race,
  aes(
    x = pct,
    y = race_short,
    fill = race_short
  )
) +
  geom_segment(aes(x = 0, y = race_short, xend = pct, yend = race_short), color = "grey70") +
  geom_point(shape = 21, color = "white", size = rel(3.5)) +
  facet_grid(job_category ~ .,
    scales = "free_y",
    space = "free",
    switch = "y"
  ) +
  scale_fill_manual(values = race_colors) +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/")

race_within_job_cat_dot_plot <- race_within_job_cat_dot_plot +
  scale_x_continuous(labels = percent) +
  theme(
    panel.grid.major.y = element_line(size = rel(0.075), linetype = "dashed"),
    strip.background.y = element_rect(fill = "white", color = "grey80"),
    axis.ticks.x = element_line(size = rel(0.5)),
    strip.text.y = element_text(angle = 180, face = "bold", size = rel(1.15)),
    strip.placement = "outside",
    axis.text = element_text(face = "bold", size = 10),
    panel.background = element_rect(fill = NA, color = "gray80"),
    legend.position = "none"
  )

race_within_job_cat_dot_plot <- race_within_job_cat_dot_plot + theme(
  panel.border = element_rect(color = "grey90", fill = NA),
  axis.text.y = element_text(size = rel(0.8)),
  axis.title = element_blank(),
  plot.caption = element_text(family = "", size = 8, hjust = 0),
  plot.caption.position = "plot"
)

race_within_job_cat_dot_plot
####################################################################################################



####################################################################################################
## A dot chart of job category distribution within race/ethnicity

race_job_cat_distribution_within_jobcat <- ungroup(race_job_cat_distribution_within_jobcat) %>%
  mutate(
    race_short = factor(race_short, ordered_lvls_race),
    job_category = factor(job_category, rev(ordered_lvls_job_cat))
  )

job_cat_within_race_dot_plot <- ggplot(
  race_job_cat_distribution_within_jobcat,
  aes(
    x = pct,
    y = job_category,
    fill = job_category
  )
) +
  geom_segment(aes(x = 0, y = job_category, xend = pct, yend = job_category), color = "grey70") +
  geom_point(shape = 21, color = "white", size = rel(3.5)) +
  facet_grid(race_short ~ .,
    scales = "free_y",
    space = "free",
    switch = "y"
  ) +
  scale_fill_manual(values = job_cat_colors) +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/")

job_cat_within_race_dot_plot <- job_cat_within_race_dot_plot +
  scale_x_continuous(labels = percent) +
  theme(
    panel.grid.major.y = element_line(size = rel(0.075), linetype = "dashed"),
    strip.background.y = element_rect(fill = "white", color = "grey80"),
    axis.ticks.x = element_line(size = rel(0.5)),
    strip.text.y = element_text(angle = 180, face = "bold", size = rel(1.2)),
    strip.placement = "outside",
    axis.text = element_text(face = "bold", size = 10),
    panel.background = element_rect(fill = NA, color = "gray80"),
    legend.position = "none"
  )

job_cat_within_race_dot_plot <- job_cat_within_race_dot_plot + theme(
  panel.border = element_rect(color = "grey90", fill = NA),
  axis.text.y = element_text(size = rel(0.9)),
  axis.title = element_blank(),
  plot.caption = element_text(family = "", size = 8, hjust = 0),
  plot.caption.position = "plot"
)

job_cat_within_race_dot_plot
####################################################################################################


####################################################################################################
## Side-by-side bar charts of job categories and ethnicity/race distribution by gender

it_data_clean <- read_csv("tech_diversity_cleaned.csv")


#  https://blog.datawrapper.de/gendercolor/
# https://stackoverflow.com/questions/17083362/colorize-parts-of-the-title-in-a-plot
# https://stackoverflow.com/questions/52902946/using-unicode-characters-as-shape

gender_ratio_by_job_cat_role <- it_data_clean %>%
  group_by(job_category, race_short, gender) %>%
  summarize(total = sum(count)) %>%
  group_by(job_category, gender) %>%
  mutate(pct = total / sum(total))


gender_ratio_by_job_cat_role <- ungroup(gender_ratio_by_job_cat_role) %>%
  mutate(
    race_short = factor(race_short, rev(ordered_lvls_race)),
    job_category = factor(job_category, ordered_lvls_job_cat)
  )

gender_job_cat_race_bar <- ggplot(
  gender_ratio_by_job_cat_role,
  aes(y = race_short, x = pct, group = gender, fill = gender)
) +
  geom_bar(stat = "identity", position = "dodge", width = 0.7) +
  scale_fill_manual(values = c("Female" = "#8700f9", "Male" = "#00c4aa")) +
  scale_x_sqrt(
    labels = scales::percent_format(accuracy = 1),
    limits = c(0, .8),
    breaks = c(0, 0.05, .1, .3, .5, .7),
    expand = c(0, 0)
  ) +
  facet_wrap(job_category ~ .)


gender_job_cat_race_bar <- gender_job_cat_race_bar +
  ggtitle(
    label = "Job categories and ethnicity/race distribution by gender",
    subtitle = paste(
      "<b style='color:#8700f9'>\u25A1 Female</b>",
      "<b style='color:#00c4aa'>\u25A1 Male</b>"
    )
  ) +
  labs(caption = "Note: The x-axis is transformed using the square root function to see smaller values. Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/") +
  geom_vline(xintercept = c(0, 0.05, .1, .3, .5, .7), color = "grey98") +
  theme_wsj() +
  theme(
    strip.text.x = element_text(face = "bold", size = 12),
    strip.background = element_rect(fill = "grey98"),
    panel.background = element_rect(fill = "grey98"),
    plot.background = element_rect(fill = "grey98"),
    panel.grid.major = element_blank(),
    panel.spacing = unit(0.8, "cm"),
    legend.position = "none",
    legend.title = element_blank(),
    plot.title = element_text(size = 14, family = ""),
    plot.title.position = "plot",
    plot.subtitle = ggtext::element_markdown(
      lineheight = 1.1,
      family = "Arial Unicode MS",
      size = 12
    ),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot"
  )

annotation_df <- data.frame(
  label = "Of all female executives,\nBlack females are about\n 2% of them, and of all\nmale executives, Black males\nare about 1% of them",
  x = .16,
  y = 1.8,
  gender = "Male",
  job_category = "Executives",
  race_short = "Black"
)

gender_job_cat_race_bar <- gender_job_cat_race_bar +
  geom_curve(
    data = data.frame(x = .02, y = 2, xend = 0.15, yend = 2, gender = "Male", job_category = "Executives"),
    aes(x = x, y = y, xend = xend, yend = yend),
    curvature = 0,
    arrow = arrow(angle = 10, ends = "first", type = "closed", length = unit(0.12, "inches"))
  ) +
  geom_text(
    data = annotation_df,
    aes(x = x, y = y, label = label),
    size = rel(3.5),
    hjust = 0,
    family = "sans",
  )


gender_job_cat_race_bar
####################################################################################################


####################################################################################################
## Dot charts of job categories and ethnicity/race distribution by gender

gender_ratio_by_job_cat_role <- it_data_clean %>%
  group_by(job_category, race_short, gender) %>%
  summarize(total = sum(count)) %>%
  group_by(job_category, gender) %>%
  mutate(pct = total / sum(total))


gender_ratio_by_job_cat_role <- ungroup(gender_ratio_by_job_cat_role) %>%
  mutate(
    race_short = factor(race_short, rev(ordered_lvls_race)),
    job_category = factor(job_category, ordered_lvls_job_cat)
  )


gender_job_cat_race_dot <- ggplot(
  gender_ratio_by_job_cat_role,
  aes(y = race_short, x = pct, group = gender, fill = gender)
) +
  geom_point(shape = 21, color = "grey80", size = 4, alpha = 0.9) +
  scale_fill_manual(values = c("Female" = "#8700f9", "Male" = "#00c4aa")) +
  scale_x_continuous(
    labels = scales::percent_format(accuracy = 1),
    limits = c(0, .8),
    breaks = seq(from = 0, to = 8, by = 2) / 10
  ) +
  facet_wrap(job_category ~ ., ncol = 1)


gender_job_cat_race_dot <- gender_job_cat_race_dot +
  ggtitle(
    label = "Job categories and ethnicity/race distribution by gender",
    subtitle = paste(
      "<b style='color:#8700f9'>\u25EF Female</b>",
      "<b style='color:#00c4aa'>\u25EF Male</b>"
    )
  ) +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/") +
  theme_wsj() +
  theme(
    strip.text.x = element_text(face = "bold", size = 12),
    strip.background = element_rect(fill = "grey98"),
    panel.background = element_rect(fill = "grey98"),
    plot.background = element_rect(fill = "grey98"),
    panel.spacing = unit(0.8, "cm"),
    legend.position = "none",
    legend.title = element_blank(),
    plot.title = element_text(size = 14, family = ""),
    plot.title.position = "plot",
    plot.subtitle = ggtext::element_markdown(
      lineheight = 1.1,
      family = "Arial Unicode MS",
      size = 12
    ),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot"
  )

annotation_df <- data.frame(
  label = "Of all female managers,<br>about 62% are white,<br> and of all male managers,<br> about 65% are white",
  x = .36,
  y = 3,
  gender = "Male",
  job_category = "Managers",
  race_short = "Asian"
)

curve_ann_df <- data.frame(x = .55, y = 3, xend = 0.64, yend = 4.5, gender = "Male", job_category = "Managers")

curve_ann_tiny_line_df <- data.frame(xmin = .61, y = 4.5, xmax = 0.67, gender = "Male", job_category = "Managers")

gender_job_cat_race_dot <- gender_job_cat_race_dot +
  geom_curve(
    data = curve_ann_df,
    aes(x = x, y = y, xend = xend, yend = yend),
    curvature = .5
  ) +
  geom_errorbar(
    data = curve_ann_tiny_line_df,
    aes(xmin = xmin, y = y, xmax = xmax),
    inherit.aes = F,
    width = 0.45
  ) +
  ggtext::geom_richtext(
    data = annotation_df,
    aes(x = x, y = y, label = label),
    size = rel(3.2),
    hjust = 0,
    family = "sans",
    label.color = "grey98",
    fill = "grey98"
  )

gender_job_cat_race_dot
####################################################################################################



####################################################################################################
## Dot charts of job categories and ethnicity/race distribution by gender
gender_within_job_cat <- it_data_clean %>%
  group_by(race_short, job_category, gender) %>%
  summarize(total = sum(count)) %>%
  group_by(job_category) %>%
  mutate(pct = total / sum(total))

gender_within_job_cat <- ungroup(gender_within_job_cat) %>%
  mutate(
    race_short = factor(race_short, rev(ordered_lvls_race)),
    job_category = factor(job_category, ordered_lvls_job_cat)
  )

gender_within_job_cat_dot_plt <- ggplot(gender_within_job_cat, aes(y = race_short, x = pct, group = gender, fill = gender)) +
  geom_line(aes(group = race_short), color = "grey80") +
  geom_point(shape = 21, color = "grey80", size = 4, alpha = 0.9) +
  scale_fill_manual(values = c("Female" = "#8700f9", "Male" = "#00c4aa")) +
  scale_x_continuous(labels = scales::percent_format()) +
  facet_wrap(job_category ~ ., ncol = 1)


gender_within_job_cat_dot_plt <- gender_within_job_cat_dot_plt + ggtitle(
  label = "Job categories and ethnicity/race distribution by gender",
  subtitle = paste(
    "<b style='color:#8700f9'>\u25EF Female</b>",
    "<b style='color:#00c4aa'>\u25EF Male</b>"
  )
) +
  theme_wsj() +
  labs(caption = "Source: Reveal, https://www.revealnews.org/topic/silicon-valley-diversity/") +
  theme(
    panel.background = element_rect(fill = "grey98"),
    plot.background = element_rect(fill = "grey98"),
    panel.spacing = unit(0.8, "cm"),
    legend.position = "none",
    legend.title = element_blank(),
    plot.title = element_text(size = 14, family = ""),
    plot.title.position = "plot",
    plot.subtitle = ggtext::element_markdown(
      lineheight = 1.1,
      family = "Arial Unicode MS",
      size = 12
    ),
    panel.grid.major = element_blank(),
    plot.caption = element_text(family = "", size = 8, hjust = 0),
    plot.caption.position = "plot"
  )


gender_within_job_cat_dot_plt <- gender_within_job_cat_dot_plt +
  theme(
    strip.text.x = element_text(face = "bold", size = 12),
    strip.placement = "outside",
    strip.background = element_rect(fill = "grey98", color = "grey40")
  )

gender_within_job_cat_dot_plt <- gender_within_job_cat_dot_plt +
  geom_curve(
    data = data.frame(x = .1, y = 4, xend = .18, yend = 2.5, job_category = "Executives", gender = "Male", race_short = "Asian"),
    aes(x = x, y = y, xend = xend, yend = yend),
    arrow = arrow(angle = 20, ends = "first", type = "closed", length = unit(0.1, "inches"))
  ) +
  geom_text(
    data = data.frame(
      label = "Of all the executives,\n4.5% are Asian women,\nand 16.3% are Asian men.",
      x = .18,
      y = 2.5,
      gender = "Male",
      job_category = "Executives",
      race_short = "Asian"
    ),
    aes(x = x, y = y, label = label),
    hjust = 0
  )

gender_within_job_cat_dot_plt
####################################################################################################
Code language: R (r)

The post Pie Chart vs. Bar Chart appeared first on nandeshwar.info.

In this article, you will learn how to create a really cool data visualization that appeared in the Economist. This chart looks like a map, but instead of your typical filled in maps a.k.a. choropleths, you see an area plot where a state should be. This chart gives you a lot of information in a small space. For example, you can see the changes in the number of cases by time. You can see the result of 2016 presidential election. And, you also see a legend to see the dates when key decisions were made.

I wanted to see whether I could re-create this chart in R. In this video, you will learn about those steps. And the great thing is about this script is you can modify it to create any map type of a plot. We could use fewer choropleths.

Let’s get started.

First, let’s load the libraries.

library(dplyr)
library(ggplot2)
library(tidyverse)
library(ggthemes)
library(scales)
library(lubridate)
library(readxl)
Code language: R (r)

Then, load an Excel file containing the location of each of the states in an 8 X 11 grid with one square per state.

state_loc_temp_file <- tempfile()
download.file("https://www.dropbox.com/s/flwu7ahky9lhlji/us-states-grid-number.xlsx?raw=1", state_loc_temp_file)

state_loc <- read_excel(path = state_loc_temp_file, col_names = TRUE, range = "A1:B51")
unlink(state_loc_temp_file)
head(state_loc)
Code language: R (r)

## # A tibble: 6 x 2
##   state boxnumber
##        
## 1 AL           73
## 2 AK           78
## 3 AZ           57
## 4 AR           60
## 5 CA           45
## 6 CO           47

Then, let's generate some fake data for each of the states. The data frame contains a metric for each state for months from February to June. One metric per month per state.

imp_data_df <- data.frame(state = rep(state.abb, 5),
                          stat_date = rep(seq.Date(from = as.Date("2020-02-01"), 
                                                   to = as.Date("2020-06-01"), 
                                                   by = "month"), 
                                          each = 50),
                          stat = rnorm(250, mean = 1000, sd = 150)) %>%
  mutate(stat = ifelse(stat > 1000 | stat < 0, 
                       sample(runif(100), n(), replace = TRUE) * 100, 
                       stat))

head(imp_data_df)
Code language: R (r)

##   state  stat_date      stat
## 1    AL 2020-02-01 782.01548
## 2    AK 2020-02-01 925.37455
## 3    AZ 2020-02-01 937.37239
## 4    AR 2020-02-01  73.23171
## 5    CA 2020-02-01 968.69072
## 6    CO 2020-02-01  59.34878

One feature of this chart is the legend below the chart that shows the changes in the metric with some key dates. Therefore, I generated a data frame with start dates, end dates, and a key decision date. Once a start date for each of the state was generated, I added some random days to get an end date as well as a key decision date.

decision_dates_df <- data.frame(
  state = state.abb,
  start_dt = sample(
    x = as.Date("2020-02-01") + months(0:2),
    size = 50,
    replace = TRUE
  )
) 

decision_dates_df <-  mutate(decision_dates_df,
    end_dt = start_dt + months(sample(2:4, size = 1)),
    end_dt = if_else(end_dt > as.Date("2020-06-01"), as.Date("2020-06-01"), end_dt),
    easing_dt = start_dt + days(sample(
      20:40,
      size = 1, replace = TRUE
    ))
  )

head(decision_dates_df)
Code language: R (r)

##   state   start_dt     end_dt  easing_dt
## 1    AL 2020-03-01 2020-06-01 2020-04-03
## 2    AK 2020-03-01 2020-06-01 2020-04-03
## 3    AZ 2020-03-01 2020-06-01 2020-04-03
## 4    AR 2020-03-01 2020-06-01 2020-04-03
## 5    CA 2020-03-01 2020-06-01 2020-04-03
## 6    CO 2020-02-01 2020-06-01 2020-03-05

Then, let's create a tibble to randomly assign the highlight colors for each of the states. I joined this table with the state locations we imported in the beginning.

plots_tbl <- tibble(
  state = state.abb,
  highlight_color = sample(c("#cd6b61", "#578ca4"),
    size = 50,
    replace = TRUE
  )
)

head(plots_tbl)

plots_tbl <- left_join(plots_tbl, state_loc)
Code language: R (r)

## # A tibble: 6 x 2
##   state highlight_color
##              
## 1 AL    #cd6b61        
## 2 AK    #cd6b61        
## 3 AZ    #cd6b61        
## 4 AR    #cd6b61        
## 5 CA    #578ca4        
## 6 CO    #578ca4

Let’s create a function to create the range bar based on these dates. This chart is dot plot or a point to show the easing date and a segment plot to create a line between start date and end date. Then I plot the two vertical lines at the end using segment annotations.

rangebar_plot <- function(df, xaxis_st_dt = as.Date("2020-02-01"), xaxis_end_dt = as.Date("2020-06-01")) {
    p <- ggplot(df, aes(x = easing_dt, y = 1)) +
      geom_point(size = 0.1) +
      scale_x_date(
        limits = c(xaxis_st_dt, xaxis_end_dt),
        date_breaks = "1 month"
      )

    p <- p + geom_segment(
        data = df,
        aes(x = start_dt,
            xend = end_dt,
            y = 1,
            yend = 1),
        size = 0.1) +
      scale_y_continuous(limits = c(0.98, 1.02), expand = c(0, 0))

    segment_st_pos <- 0.99

    p <- p + annotate(
        "segment",
        x = df$start_dt,
        xend = df$start_dt,
        y = segment_st_pos,
        yend = segment_st_pos + .02,
        size = 0.1
      ) +
      annotate(
        "segment",
        x = df$end_dt,
        xend = df$end_dt,
        y = segment_st_pos,
        yend = segment_st_pos + .02,
        size = 0.1
      )

    p <- p + theme_void()

    p
  }

rangebar_plot(filter(decision_dates_df, state == 'CA'))
Code language: R (r)

Next, let's create a function to create an area graph which will also show the state name and a line with the highlight colors of our choosing as seen in the Economist article for the 2016 presidential election results.

state_plot <- function(df, 
                       highlight_color = "blue", 
                       xaxis_st_dt = as.Date("2020-02-01"), 
                       xaxis_end_dt = as.Date("2020-06-01")) {
    g <- ggplot(df, aes(x = stat_date, y = stat)) +
      geom_area(fill = "#559ab7") +
      scale_y_continuous(limits = c(0, 1100), expand = c(0, 0)) +
      scale_x_date(limits = c(xaxis_st_dt, xaxis_end_dt),
                   date_breaks = "1 month") 
    
    g <- g +
      annotate(
        "text",
        x = as.Date("2020-02-01"),
        y = 1070,
        label = unique(df$state),
        size = 1.5,
        color = highlight_color,
        hjust = 0,
        vjust = 1,
        fontface = "bold"
      ) 
    
    g <- g +
      geom_hline(yintercept = Inf,
                 size = 0.3,
                 color = highlight_color) 
    
    g <- g +
      theme(
        axis.ticks.length.x = unit(1.3, "points"),
        axis.ticks.x = element_line(color = "#8aa6b6", size = 0.2),
        axis.line.x = element_line(color = "#8aa6b6", size = 0.2),
        axis.text = element_blank(),
        axis.title = element_blank(),
        panel.background = element_rect(fill = "#d5e4eb", linetype = 0),
        panel.border = element_blank(),
        plot.background = element_rect(fill = "#d5e4eb",
                                       color = NA),
        panel.grid = element_blank(),
        axis.ticks.y = element_blank()
      )
    
    g <- g + geom_hline(yintercept = 500,
                        size = 0.1,
                        color = "white")
    return(g)
  }

state_plot(filter(imp_data_df, state == "CA"))
Code language: R (r)

Next, rather than plotting everything, we actually store the plots in the tibble we created earlier. This way, we still retain each state, its location in the grid and the associated plot in a tiblle that we can manipulate.

We use the map2 function to iterate over the state and the highlight color to create the area plot, the error bar plot, and put them into a plot using the plot_grids function from the cowplot package. We could have used a loop here, but mutate and map2 let us create a variable to store the ggplot object.

plots_tbl <- mutate(plots_tbl,
                    plots = map2(.x = state, .y = highlight_color, 
                                 .f = function(x, y) { 
                                   st_plt <- state_plot(filter(imp_data_df, state == x), 
                                                        highlight_color = y)
                                   rng_plt <- rangebar_plot(filter(decision_dates_df, state == x))
                                   cowplot::plot_grid(st_plt, rng_plt, nrow = 2,
                                                      labels = NULL,
                                                      align = "v",
                                                      axis = "t",
                                                      rel_heights = c(5, 1))
                                   }
                                 ))

head(plots_tbl)
Code language: R (r)

## # A tibble: 6 x 4
##   state highlight_color boxnumber plots 
##                    
## 1 AL    #cd6b61                73   
## 2 AK    #cd6b61                78   
## 3 AZ    #cd6b61                57   
## 4 AR    #cd6b61                60   
## 5 CA    #578ca4                45   
## 6 CO    #578ca4                47 

Next, let’s create the legend plot that’s explains the area plot. Since it has additional information, we can’t use the previous function and we need to create a separate plot.

lp <- ggplot(filter(imp_data_df, state == "CA"), aes(x = stat_date, y = stat)) 
lp <- lp + geom_area(fill = "#559ab7") +
  scale_y_continuous(limits = c(0, 1100), expand = c(0, 0), labels = comma, position = "right") +
  scale_x_date(breaks = as.Date("2020-06-01") - months(0:4), 
               labels = rev(c("F", "M", "A", "M", "Jun")),
               expand = c(0, 0)) 
lp <- lp + geom_hline(yintercept = Inf, size = 0.3, color = '#578ca4') 

lp <- lp + theme(axis.ticks.length.x = unit(1.5, "points"),
                 axis.ticks.x = element_line(color = "#8aa6b6", size = 0.2),
                 axis.line.x = element_line(color = "#8aa6b6", size = 0.2),
                 axis.text = element_text(color = "black", size = 2.2),
                 axis.title = element_blank(),
                 panel.background = element_rect(fill = "#d5e4eb", linetype = 0),
                 panel.border = element_blank(),
                 plot.background = element_rect(
                   fill = "#d5e4eb",
                   colour = NA
                   ),
                 panel.grid = element_blank(),
                 axis.ticks.y = element_blank())

lp <- lp + geom_hline(yintercept = 500, size = 0.1, color = "white")

lp
Code language: R (r)

Next, let’s create the legend plot for the highlight color. In this case, we are creating a legend plot for pizza preference.

pizza_pref <- data.frame(y = 1,
                         x = 0,
                         label = c("Thin", "Deep"),
                         color = c('#cd6b61', '#578ca4'),
                         stringsAsFactors = FALSE)

pizza_pref_plot_fn <- function(what_type) {
  df <- filter(pizza_pref, label == what_type)
  label_color <- df$color
  
  g <- ggplot(data = df,
              aes(x = x, y = y, label = label, color = label)) +
    geom_segment(aes(xend = 1, yend = y), size = 0.3) 
  g <- g + geom_text(size = 1.5, color = label_color,
                     hjust = 0,
                     vjust = 1.5,
                     fontface = "bold") 
  g <- g + scale_color_manual(values = label_color) +
    theme_void() +
    theme(legend.position = "none")
  
  g
}

pizza_pref_plot_fn(what_type = "Deep")
Code language: R (r)

We need to create another legend for the error bar type of a plot, which shows the start, end and the easing dates.

range_bar_legend_plt <- filter(decision_dates_df, state == 'WY') %>% {
  rangebar_plot(df = .) + 
    scale_x_date(limits = c(as.Date("2020-02-01"), as.Date("2020-07-01")), date_breaks = "1 month") + 
    annotate("text", x = .$start_dt, label = "Start Date", y = 1, hjust = 1, size = 1) + 
    annotate("text", x = .$end_dt, label = "End Date", y = 1, hjust = 0, size = 1)  + 
    annotate("text", x = .$easing_dt, label = "Easing starts", y = 1, vjust = 1.2, size = 1)
}
  
range_bar_legend_plt
Code language: R (r)

Final Touches

We’re getting to the final finishing touches now.

Let’s create an empty vector for the 8 X 11 grid cells.

my_list <- rep(NA, 88)
Code language: R (r)

Then, in this vector let’s store the plots in their respective locations.

my_list[plots_tbl$boxnumber] <- plots_tbl$plots
Code language: R (r)

Place the legend in the first box.

my_list[[1]] <- lp
Code language: R (r)

And then the preference legends in the third and fourth box.

my_list[[3]] <- pizza_pref_plot_fn(what_type = "Thin")
my_list[[4]] <- pizza_pref_plot_fn(what_type = "Deep")
Code language: R (r)

Now, as a final step, place all the subplots in a grid using the plot_grid function from cowplot.

gridded_plots <- cowplot::plot_grid(plotlist = my_list, nrow = 8, ncol = 11)  
gridded_plots
Code language: R (r)

Finally, let’s add title, subtitle and captions.

title_pos <- 0.01


final_plot <- cowplot::ggdraw(gridded_plots, ylim = c(-0.05, 1.1)) +
  cowplot:: draw_plot(range_bar_legend_plt, x = .35, y = .915, width = .4, height = 0.04) +
  cowplot::draw_label("Important dates to remember", x = .48, y = 0.95,  hjust = 0, vjust = 0, size = 4) +
  cowplot::draw_label("States of play", x = title_pos, y = 1.1,  hjust = 0, vjust = 1, size = 8, fontface = "bold") +
  cowplot::draw_label("Random data. Change people's opinion using this chart", x = title_pos, y = 1.065,  hjust = 0, vjust = 1, size = 5.5) +
  cowplot::draw_label("https://www.nandeshwar.info. Generated using R", x = title_pos, y = -0.05,  hjust = 0, vjust = -1, size = 4) +
  cowplot::draw_label("Pizza preference surveyed in 2019", x = .186, y = 0.95,  hjust = 0, vjust = 0, size = 4)

Code language: R (r)

And save it using the ggsave function! And done!

ggsave(plot = final_plot,
       filename = "my_us_map_plot.png",
       width = 6,
       height = 4, 
       bg = "#E5EBF0") # changing the whole background color
Code language: R (r)

Compare it with the original plot. Pretty close! I am happy with these results, because now this script can be used for other data sets and additional graphic design would be minimal.

There you have it! A map-like plot with an individual area plot for each state, along with some additional details for each of the plots, legends and other keys.

Let me know what you think. And also, if anything is unclear, please let me know.

Video Walkthrough

The post How to Create an Economist Data Visualization of US Map Using R appeared first on nandeshwar.info.

]]>