Pivot Tables in R

For myself and many colleagues, adopting R as our main analysis tool was only after years of using other products like SPSS, Stata, or Excel. Lets admit it – Excel is all too often what many of us use for initial data inspection, preprocessing, and some descriptive statistics. A common step is to create a quick pivot table in Excel to explore some of the different variables in the dataset. The idea of a pivot table is to use two variables to group data and then summarize their relationship.

For example, I often work with voter files that include voter history. I often want to group by the election on one axis and the ballots cast by party, age, geography, or some other available variable. My old habits have me drop this data into Excel and make a pivot table to summarize these variables. Something that I kept wanting to do with R is replicate this tool.

To create a similar tool in R requires a little script and dplyr. Here is what you need to do. First decide what the variable you will group on for the rows, then what variable to group on for the columns. Finally, decide how you want to measure the interaction between the rows and columns. Using dplyr’s summarise function, you can count, sum, calculate means or medians, and so on.

Here is how the script is set up. If you use it – change out “data” with your dataframe, and “rows” with the variable name you want to use, and “columnvariable” for the variable to use for the columns. How this works is

#Libraries Required
library(pander)  #Optional if you want the last line to run

pivot <- data %>% group_by(rows) %>% summarise(n=n())  #Data is grouped to create the row titles you want

#Start summarizing by the rows

columns <- unique(columnvariable)  #The unique values of the column variable are stored for use in the loop.

#Now, loop through the column variable to create summaries.
#Change the summarise function as desired.

for (i in seq_along(row_category)) {
  temp <- data %>% group_by(row) %>% filter(column==columns[i]) %>% summarise(summary=n()) 
  names(temp)[names(temp)=="summary"] <- columns[i] #Renames the summarized column with the particular column name to use in join.
  pivot <- full_join(pivot, temp, by="row") #Adds the results for each analysis to make the new table. 

rm("temp") #Delete the temp file for creating summary

#Be fancy and create markdown table result for a nice report!

pandoc.table(pivot, style="rmarkdown")

You can also add more summary functions to the loop, just remember to adjust the renaming step to address the additional summary columns. The last line in this script makes a table that can be used in Markdown or RMarkdown.


The Policy Unicorn: Or why I am voting Yes on Measure 97

This election seems to be about imperfect choices. Nothing seems to fit the expectations of an increasingly agitated public that is sure in its values, and is disappointed to see them not reflected in the options presented to them. This is not just on the national stage, but also on local and statewide issues. (For example David Roberts’s excellent analysis of fractured progressive politics in Washington State captures this story well.)

Its out there somewhere! (Source: ZehnKatzen Times)

Imperfect choices present themselves in Oregon too with Measure 97. This measure is on the face of it a simple one: raise the corporate tax rates in Oregon to address both a relatively low corporate tax rate and to address years of budget shortfalls. The Measure seeks to do this by creating a new tax rate of 2.5% tax on companies with sales over $25 million. The Measure also focuses solely on one type of corporation: the C corporation. This last point is key.

There are a number of options the owners of corporations can chose from to create their company. C corporations are the simplest. But there are many reasons why the owners of a company do not want a C corporation. The key reason is taxation. C corporations are taxed on their own earnings and then again when they distribute earnings to owners. So many smaller business seek other corporate forms, such as S corporations or LLC. (For more details, check out this article.)

So why would you want to be a C corporation then? Because you want to have more than 100 shareholders – or in other words, want to be publicly traded. You might also want it for other outside investment opportunities where growth is key. (Note: a C corporation might not be public, but might have plans to do so.) This is one spot where Measure 97 creates a problem. A company like Kroger, which we know as Fred Meyers, will be taxed. Safeway will not be taxed because it was taken private in 2014 by Cerberus Capital, and therefore is in a different corporate status. Same business, same products, one will pay 2.5% and the other won’t. In essence Measure 97 gives private equity a free pass to operate in Oregon.

Based on this last point – you might think this Measure is dangerous. I’d normally agree. By exempting private equity firms, we are rewarding yet again the earnings that Mitt Romney and others pile up. (These firms often also distribute their earnings as capital gains to owners, which is taxed much lower than their income tax rates.)

But here is the larger problem: Oregon taxation is broken. The legislature has chosen or failed to address it since the great crises of Measures 5 and 50 that changed our tax landscape. Corporate tax rates have slipped in part because corporations bring more finance expertise to the game than our citizen legislators. Its not even close to an even playing field. Like so many issues in this election cycle we are paralyzed by a desire to find our unicorns. Mythical creatures that sadly do not exist, but still capture our imagination.

Instead, the failings of Measure 97 buoy my hopes. First, by having an uneven application across corporations, it assures there will not be big price swings for the consumer. Kroger will not raise prices when its competitors are not subject to the tax. Second, it is a clear message that something must be done. Measure 97 will likely require a legislative fix. That might be worrying to some, but I am optimistic. Any legislative engagement on tax issues is better than nothing at this point. Finally, it is a chance to remind businesses that they operate in Oregon under a social license. They operate because they are part of a system to provide goods and services. Profit is a reward for this – but it should not be out of sync with the public benefits a well functioning market brings. If profits grow, but the public suffers, that license should be up for review.

For these reasons I will vote yes on Measure 97 and hope you do to.



Making Sense of Rangeland Politics

The past decade has had many worried about fate of the Greater Sage Grouse. The bird’s population has been declining, impacted by habitat fragmentation and industrialization of the landscape. This is another classic natural resource conflict similar to the salmon and Spotted Owl conflicts of the 1980’s, 1990’s – and well now. Issues at hand are questions about the science of the species (how they breed), the role of humans in fire management, and how different rangeland users impact the bird population differently. At the same time, large record setting catastrophic wildfires have been impacting the sage brush sea – adding more pressure to the sage grouse population. It is this growing set of issues that provide a backdrop for the antics in Burns, Oregon.


Steens Mountain

Federal laws tend to be better sledgehammers than scalpels, something many of us in natural resource management are aware of. As policy researchers, many of us become excited when we hear about collaborative efforts to manage regulations in a cooperative way. Simply shutting down a resource never produces the best outcomes for species or people. I had a chance to visit Burns in 2013 and tour with local managers and ranchers to explore the wildfire and sage grouse issues. The big questions then on the range was how to manage fire and the sage grouse? Was cattle grazing an appropriate way to reduce fuel loads? What are we to do with encroaching Juniper forests that are displacing sage grouse and adding to fire threats? How can families continue to ranch and pass land on to children and not run afoul of tax law?


Spring Branding in Harney County

Some answers were emerging. New research from the Eastern Oregon Agricultural Research Station was leading to testing of a reseeding technique for sage brush that could make wildfire restoration more effective. This was a development many in the community were proud of. Ranchers in some cases were investing in their own research to test grazing patterns. Like many resource dependent communities such as fishers or loggers, they have an intimate understanding of environmental response to human activities. Though sometimes these understandings can be compromised by economic pressures. But these issues were being worked on as a community together.

But not everyone follows the rules. The Hammonds clearly decided to repeatedly flaunt the law, and not for such pure reasons as some argue. It is their actions that activated one of those federal sledgehammer laws. But it was not one many would have anticipated. It was not the Endangered Species Act that set this one off. (The bird was recently found not to warrant federal protection.) It was not ostensibly grazing or rangeland management laws (though that is a backdrop.) It was the Antiterrorism and Effective Death Penalty Act of 1996 that set this one off.


The Pete French Round Barn from the 1870’s

This law set mandatory minimum sentences for destroying federal property with arson. Some have argued that this was a response to ‘ecoterrorism’ – but the law was signed before many of those examples. It was signed a year after the Oklahoma City bombing, a domestic terror attack. So to see it applied in this case has raised eyebrows. Is a five year sentence for wildfires versus burning a federal facility fair? To be honest, considering the threats to wildland fire fighters, I am not one to take any form of arson lightly. Those firefighters have had a terrible series of seasons in the West, and fire is not something to take into one’s own hands.

Birds and Cattle - the Western Refuge Way

Birds and Cattle – the Western Refuge Way

At the same time mandatory minimums and antiterror law are massively disproportionately applied to communities of color. These types of laws have fueled incarcerations that we have a hard time seeing as just. The Hammonds are a rare case of the white guys getting the book thrown at them. But the militia (terrorist) response in Burns has been triggered by the application of the law. This law is one many of us protest, but not with arms. (And militia movements are tied to lots of other ugly things in the West.) Privilege allows these individuals to waltz around a federal office with guns.

But we need to parse out three issues here: the role of federal antiterror law and more specifically mandatory minimum sentences, the history of conflict in western resource management, and white terror facilitated by a culture and law of permissive weapon ownership. These three issues are meeting in Burns, and they carry a huge burden with them.

Today’s headlines are sad because there was a seed of cooperation clearly taking root in the region when I visited. Trust has been hard to find between the environmentalists, ranchers, land managers and scientists. There was progress in working towards a more nuanced understanding of rangeland management and ecology. After the militia are gone – hopefully without harm or providing them the attention they crave – the underlying issues will remain. Can these efforts be returned to in the aftermath? The armed individuals do not speak for the local community and I hope we remember that.

If you want to read more on the history of the Malheur, consider the book by Nancy Langston: “Where Land and Water Meet.” Its an amazing history that shares the seesawing priorities in land management and is very well written.


How Much is Too Much?: Bikes, Technology and Death

A recent Willamette Week piece questioned the nature of crisis that is being invoked to raise support for street fees. The recent policy debates on new funding options for City’s street network are in part based on a concern that we are falling behind in keeping streets safe. To this the Willamette Week notes:

“There’s no clear evidence that driving, walking or bicycling in Portland is growing more dangerous. State numbers show traffic deaths in the city haven’t changed dramatically over the past decade. In fact, Portland is rated as one of the nation’s safest cities for pedestrians and bicyclists.”

The difficulty in this claim is its cool dismissal of the baseline of loss. There have been many tragic deaths on our roads, and some have caused change. These deaths do seem to be focusing more on those who walk along our streets, as those on bikes seem to be doing better. In 2013 Portland saw no loss of life by those on bike. But there remains an underlying series of cultural and technological problems for those not traveling in automobiles or other larger vehicles. Being on bike or foot is to travel with an asymmetrical power relationship with the other users on the road – the auto occupants are likely to walk away after a collision with a pedestrian or bike.

The question we face with these debates is the difficult question: How many lives are too many to lose? How much do we spend to save a life? How do we explain the types of death: careless, reckless, purposeful…?

[continue reading…]


Public as Customer: Fees, Opinions and Equity

Portland and Oregon have been stuck in a revenue trap for over a decade. Limitations on property tax in the 1990’s have impacted local governments heavily. While property tax growth is limited, the costs of providing local services have grown much faster. Figure 1 shows the statewide experience of local governments as the direct expenditures have grown faster than than the local revenue – causing a need for transfers from state and federal government, and creative fixes through other revenue sources.

Local Direct Expenditures versus Local Property Tax Revenues

Figure 1: Local Direct Expenditures versus Local Property Tax Revenues (Source: US Census, Annual Surveys of State and Local Finance)

The proposed Portland Street Fee is one of these creative fixes. Not a tax, not quite a fee, but a short term fix to address infrastructure maintenance issues. The proposal would raise at least $34 million a year by assessing a new fee on households and business in the city of Portland. The household fee may range from $8-$12 per month. Revenue from residences and business would each make up half of the total revenue.

[continue reading…]


How Political Institutions Influence Marine Spatial Planning

Recently I had the opportunity to explore why some democracies adopt coastal and marine spatial planning (CMSP) and others do not. I am curious if there are political factors that increase the likelihood of a nation adopting CMSP. Using a quick  quantitative comparative method, I tested if there were institutional factors that inform adoption of this policy tool. The great dataset developed by Arendt Lijphart for his book, Patterns of Democracy provides a number of variables measuring varieties of democracy. In this study, Lijphart explored 36 democracies to understand how different institutions contribute to effectiveness at policy making. He coded many variables around institutional structures that exist in these democracies. Lijphart was interested in the question of which forms of democracy perform better – with his argument that consensus forms being the more successful model.

For my questions I was more intrigued by the idea that these parameters might provide clues on the adoption of certain policies. My initial analysis suggests that there is a link between the level of decentralized power and the adoption of CMSP. In particular, political systems that enable minority interests to participate might increase the adoption of CMSP.

[continue reading…]


What is this Project About?

My goal with this writing is to try and trace the intersections of political theory, science and technology studies and the environment. It in part gives rise to the name of this site. By exploring praxis I draw attention to the need to apply our philosophical arguments on natures and societies. The hope is this is a place to explore how the construction of society in political philosophy and the construction of nature can be better explored.

[continue reading…]