Introductory R workshop

A colleague and I recently presented a 1/2 day workshop on introductory R and rQTL. We wrote the course materials in knitR, and presented the workshop at the Waite Campus in Adelaide.

It was successful, and the course materials and preparatory code are available at github.

Reducing electricity consumption to affect political change

Many people are concerned about the negative effects that humans are having on the Earth's ecosystems, and with good reason: those systems support us, and without them we would not survive. It is estimated that for every dollar of GDP produced, the natural world contributes .75c -- these are services that nature provides for free and, without which, our quality of life would drop by 75%.

A big cause of human destruction of natural systems is in the provision and delivery of energy. The main culprits are fossil fuels which are starting to alter the world's climate -- a change that, if unchecked, will wreak great destruction on both the natural and human systems that support us. There are now technologies (solar PV and wind turbines) that can produce energy with much  less damage to the environment, and it is imperative that we transition to these as soon as possible. Unfortunately, there are entrenched interests that are heavily committed to fossil fuels and use their influence to slow the adoption of newer, less harmful, methods of energy production.
What is clear is that majority political support for serious action on climate change is currently lacking.

There are huge forces at work here. In the absence of a government with the ability and disposition for serious action, what can an individual do to help this process?

Number one: Use Less

When it comes to reducing one's impact on the systems that support us, there is simply no substitute for using less. Even "green" energy sources have impacts (albeit much more minor). Using less frees money and resources that can be better deployed by helping other, less fortunate, people adapt.

Number two: Avoid electricity or fuel where possible

Electricity is hard to produce, and there are always wastes and damage to the environment -- even when using solar PV or wind turbines. This is because the manufacture, installation and maintenance of these generally result in carbon emissions and other forms of pollution.
Therefore, if a task can be performed without using electricity at all, it is preferable to do so. A great example of this is hot water, which can be heated directly from the sun without need for electricity at all. By doing this, your electricity consumption will be much lower.
Similarly, a solar cooker can cook food without fuel or electricity, using only heat directly from the sun. These are simple and cheap to build.

Number three: buy carbon-emission-free ("green") power, and/or install solar panels

Here is where your large reductions in consumption will help -- because you're now using a lot less power, you will not need many solar panels to offset your electricity consumption (cheaper!).
At my house, we have reduced our power consumption to about 3 kWh/day for a family of four without significant lifestyle sacrifices. I think it can go lower still. We have a 2 kW solar PV system, which produces an average of about 6 - 8 kWh/day (averaged over the year) -- more power than we consume. We do draw power from the grid though, as we do use power at night (when the PV panels aren't producing), which is why we also buy carbon-emission-free power. By doing this, we ensure that we're not directly supporting the use of coal for electricity generation and because our power consumption is so low, the additional cost is very small. This also supports the businesses that are producing renewable carbon-free energy in Australia.

Dirty companies only have as much money as we give them

Using less means less revenue for the large energy generation, distribution and retail companies that, generally, are slowing the adoption of renewable energy technologies, and affecting public policy in negative ways. Make no mistake -- the combined actions of individuals is very powerful. The uptake of domestic solar PV has already had a huge effect on the energy generators and retailers in Australia.
This is because solar PV is usually producing power at precisely the times that energy is most expensive (energy retailers pay a fluctuating ("spot-") price for electricity,  and the times when the spot price is highest are the most profitable for the generators. Domestic solar PV is taking the edge from this demand, which is having a significant affect on the profitability of some of the big dirty producers. These kinds of commercial disincentives work to the advantage of renewable energy technologies.

If Australians continue to reduce their consumption and simultaneously install distributed generation (like solar panels, solar hot water, etc) and buy carbon-emission-free power, this will help to encourage the necessary changes. This will have a concrete effect on Australia's carbon emissions and reduce the likelihood of catastrophic climate change.

Remember, a carbon tax only has benefits because it causes people to use less -- we as concerned citizens can use less all by ourselves.

This post was written by Angus Wallace and first appeared at guesstimatedapproximations.blogspot.com.au

filter strings with awk, extended regular expressions

I've been wanting to filter out a DNA sequence within a line of text using awk, and have been unsuccessful. I was trying:

gawk '{gsub(/[ACTG]{10,}/,""); print}'

which I expected to work, but it was not. I've found that gawk (GNU awk) has an extra setting that allows this syntax:

gawk --re-interval '{gsub(/[ACTG]{10,}/,""); print}'

This works.

Rainwater modeling

I have spent a fair amount of time thinking about water. My family and I live in Adelaide, where water is hard to come by. The energy intensity of Adelaide's water supply is about 0.8 - 1.8 kWh/kL [1], while Adelaide's new desalination plant is expected to use 3.5 - 5.5 kWh/kL [2]. Since the average household uses about 191 kL per year, the energy associated with this is between 152 - 1000 kWh per year -- a significant amount of energy.

Thus, we want to use less of this expensive resource, and more of the free water that falls on our roof and currently goes to waste. For this reason, we have installed 35 kL of rainwater tanks, and they are currently collecting from the back part of the house, the pergola and the garage. As part of my plans for developing this system, I wanted to understand what sort of performance we can expect from this system when it is completed. In other words, do we have enough storage so that we will not run out of water?

How much storage is required?

I went to the Australian Bureau of Meteorology and downloaded the all the monthly rainfall data for Adelaide [3]. Then I built a simple model that estimates our monthly average consumption. The model pretends that we installed our tanks when rainfall records began (in 1884). I then ran the model through all the data to see how our system would have performed historically. Thus we have 129 years' real data to test things.

Assumptions:

1. We collect water from every roof on our land (not currently the case, but a near- to medium-term goal)
2. We collect 85% of the water that falls on these surfaces (the rest splashes, evaporates, etc)
3. There are many assumptions about how much we use, but it seems to be approximately similar to the bills we get from our utility (although, our usage in summer may be higher than this model suggests).

Results

With our system as is (when the above assumptions are met), we would have run out of water in 100 out of 129 years.
If we capture grey water from the kitchen sink and washing machine, and use that instead of potable water for watering the garden, then we would have run out of water in 41 our of 129 years.
If in addition to the grey water, we move to a dry toilet (eg. composting) then we would have run out of water in three years.
If we increase our water storage to 45 kL, but retain the flushing toilet, we would have run out of water in 17 out of 129 years. All my modelling was performed in a simple spreadsheet that can be downloaded here [4] (the model was created in a spreadsheet in LibreOffice [5] -- a free office program that you may install for no cost if you want) -- you can play with the numbers yourself. Please let me know in comments or by email if you find any problems.

Modeled rainwater captured, in storage and consumption on a month-by-month  basis for an average year (rainfall averaged monthly since records began in 1884)

Conclusions

If the goal is not to run out of water, it is much more effective to reduce consumption than to increase storage. Also, capturing water from a larger roof area is also much more effective than increasing storage. Grey water is very powerful here, as we will be able to use water twice: once in the home (washing, etc) and then again in the garden. This is useful provided the use of grey water in the garden offsets water we would otherwise use in the garden.

Other considerations

I have also been reading about the energy cost of pumping domestic rainwater. CSIRO have published a paper on this [6], which says that having a pump that directly pressurises the pipes is a very energy expensive way to run things and results in energy consumption that is close to desalination.
I am hoping, as much as possible, to run the garden on gravity-fed water
wherever possible to mitigate this. Also, instead of having a system where a pump directly pressurises our existing water pipes in the house, I'm hoping to create a header tank up high, and then gravity feed the house [EDIT: I have decided not to do this, and have instead installed a variable speed pump. I will discuss this in a future post]. The problem is that, because we live in an urban setting, I'm limited by Council regulations how elevated such a tank can be. I will most more on this in another document.

References

[1] Climate Change and Water: International Perspectives on Mitigation...  edited by Carol Howe, Joel B. Smith, MS. Jim Henderson

[2] http://www.sawater.com.au/NR/rdonlyres/2AF55919-F858-4AB2-93E3-534E62E6DC73/0/DesalEISChapter6.pdf
[3] http://www.bom.gov.au/climate/data/
[4] https://www.dropbox.com/s/od108as4l3v4ogu/water%20calculations.ods
[5] www.libreoffice.org
[6] https://publications.csiro.au/rpr/download?pid=csiro:EP114797&dsid=DS4

This post was written by Angus Wallace and first appeared at guesstimatedapproximations.blogspot.com.au

knitR and LaTeX for bioinformatics

knitR is a great way of dynamically generating reports, and ensuring that they're always based on up-to-date data. Having the reporting closely tied to the data and code that generated it is one of the tenets of reproducible research. In bioinformatics, one often finds oneself repeatedly generating figures, graphs, etc, for multiple labels (eg. probes, genes, etc). knitR is a great fit for this.

I had some difficulty finding the best way of iterating and generating multiple reports, and the code below is the best way I have found so far.

The following example code iterates over each label in a data.frame, and generates a separate .pdf for each. In a real scenario, each label could be a gene, and we could pull out whatever data/tables/figures we wanted.

Note that you may need to install some R libraries for this code to work:

• ggplot2
• xtable
• knitr

Below is example code to do this. A control R-script. Adjust the path in the knit2pdf command, put this in a file, make it executable, and run it: The actual knitR file is below. This is essentially a LaTeX file with blobs (called "chunks" in knitR) of R in it that are evaluated and their output included in the final document, as desired. I saw the xtable technique on this blog.

Blast, updated

I have found a way of parallelising blast across a fasta file and multiple databases at the same time. I often wish to blast a .fasta file of sequences against a heap of databases, so this represents quite a speed-increase.

DBs= "/path/to/TriFLDB \
/path/to/harvardTC_Ta12"
in_fd=/path/to/input_file.fasta
out_fmt='\"6\"'

parallel --gnu "cat ${in_fd} | parallel --gnu --nice 19 --block 100k --recstart '>' --pipe ${BLAST_PATH}/${blast_type} -outfmt "\""${out_fmt}"\"" -query - -db {1} | awk '{n=split(\"{1}\",prnt, \"/\"); print \$0,prnt[n] }'" ::: ${DBs} 

I have included an awk in the pipe within parallel that appends a column with the name of the database. You can remove that if you wish, but you will then not know which database a given hit came from.

This code works by using parallel to call an instance of parallel for each database to be analysed. The sub-parallel then parallelises over the fasta file, as per my previous post on this subject.

I have been working with blast for aligning sequences. This is a fairly computationally intensive exercise that is very worth parallelising. GNU parallel is a great tool for this, but I found it rather unintuitive to use. I also didn't realise that my first implementation was not actually parallelising blast, but it was still using only one processor core (traps for the unwary!). I found an implementation that worked, though, and it's below:

cat $in_fd | parallel --gnu --max-procs ${max_cores} --block 100k --recstart '>' --pipe blastx -evalue 0.01 -outfmt 6 -db ${blast_DB_loc} -query - > output_file.dat

# ${in_fd} is a fasta-formatted input file
# ${blast_DB_loc} is the database to blast against
# ${max_cores} is the number of cores one wishes to use

Please note that parellisation like this is only appropriate when the order does not matter and the analysis of one segment does not rely on the output of the analysis of another segment.