Blog//tdi.github.io/2016-08-13T18:18:00+02:00Automatic PostgreSQL config with Ansible2016-08-13T18:18:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2016-08-13:2016/08/13/automatic-postgresql-config-with-ansible/<p>If for some reasons you can’t use dedicated DBaaS for your PostgreSQL (like AWS RDS) then you need
to run your database server on a cloud instance. In these kind of setup, when you scale up or down
your instance size, you need to adjust PostgreSQL parameters according to the changing RAM size.
There are several parameters in PostgreSQL that highly depend on RAM size. An example is
<code>shared_buffers</code> for which a rule of thumb says that is should be set to 0.25*RAM.</p>
<p>In DBaaS, when you scale the DB instance up or down, parameters are adjusted for you by the cloud
provider, e.g. AWS RDS uses parameter groups for that reason, where particular parameters are
defined depending on the size of the RAM of the RDS instance.</p>
<p>So what can you when you do not have RDS or any other DBaaS? You can always keep several
configuration files on your instance, each for a different memory size, you can rewrite you config
every time you change the size of the instance… or you can use Ansible role for that.</p>
<p>Our Ansible role will be very simple, we will have two tasks. One will change the PostgreSQL config,
the second one will just restart the database server:</p>
<div class="highlight"><pre><span></span><span class="nn">---</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Update PostgreSQL config</span>
<span class="l l-Scalar l-Scalar-Plain">template</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">src=postgresql.conf.j2 dest=/etc/postgresql/9.5/main/postgresql.conf</span>
<span class="l l-Scalar l-Scalar-Plain">register</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">pgconf</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Restart postgresql</span>
<span class="l l-Scalar l-Scalar-Plain">service</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">name=postgresql state=restarted</span>
<span class="l l-Scalar l-Scalar-Plain">when</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">pgconf.changed</span>
</pre></div>
<p>Now we need the template, where are the calculations take place. RAM size will be taken from the
Ansible’s fact called <code>ansible_memtotal_mb</code>. Since it returns RAM size in MBs, we will stick to MBs.
We will define the following parameters, you can adjust them to your needs:</p>
<ul>
<li><code>shared_buffers</code>, as 0.25*RAM size,</li>
<li><code>work_mem</code>, as <code>shared_buffers/max_connections</code>,</li>
<li><code>maintenance_work_mem</code>, as RAM GBs times 64MB,</li>
<li><code>effective_cache_size</code>, as 0.75*RAM size.</li>
</ul>
<p>For max_connections we will define a default role variable of 100 but we will allow to specify it at
a runtime. The relevant parts of the <code>postgresql.conf.j2</code> are below:</p>
<div class="highlight"><pre><span></span><span class="x"> max_connections = </span><span class="cp">{{</span> <span class="nv">max_connections</span> <span class="cp">}}</span><span class="x"> </span>
<span class="x"> shared_buffers = </span><span class="cp">{{</span> <span class="o">(((</span><span class="nv">ansible_memtotal_mb</span><span class="o">/</span><span class="m">1024.0</span><span class="o">)|</span><span class="nf">round</span><span class="o">|</span><span class="nf">int</span><span class="o">)*</span><span class="m">0.25</span><span class="o">)|</span><span class="nf">int</span><span class="o">*</span><span class="m">1024</span> <span class="cp">}}</span><span class="x">MB</span>
<span class="x"> work_mem = </span><span class="cp">{{</span> <span class="o">((((</span><span class="nv">ansible_memtotal_mb</span><span class="o">/</span><span class="m">1024.0</span><span class="o">)|</span><span class="nf">round</span><span class="o">|</span><span class="nf">int</span><span class="o">)*</span><span class="m">0.25</span><span class="o">)/</span><span class="nv">max_connections</span><span class="o">*</span><span class="m">1024</span><span class="o">)|</span><span class="nf">round</span><span class="o">|</span><span class="nf">int</span> <span class="cp">}}</span><span class="x">MB</span>
<span class="x"> maintenance_work_mem = </span><span class="cp">{{</span> <span class="o">((</span><span class="nv">ansible_memtotal_mb</span><span class="o">/</span><span class="m">1024.0</span><span class="o">)|</span><span class="nf">round</span><span class="o">|</span><span class="nf">int</span><span class="o">)*</span><span class="m">64</span> <span class="cp">}}</span><span class="x">MB</span>
<span class="x"> effective_cache_size = </span><span class="cp">{{</span> <span class="o">(((</span><span class="nv">ansible_memtotal_mb</span><span class="o">/</span><span class="m">1024.0</span><span class="o">)|</span><span class="nf">round</span><span class="o">|</span><span class="nf">int</span><span class="o">)*</span><span class="m">0.75</span><span class="o">)|</span><span class="nf">int</span><span class="o">*</span><span class="m">1024</span> <span class="cp">}}</span><span class="x">MB</span>
</pre></div>
<p>You can now run the role every time you change the instance size, and the config will be changed
accordingly to the RAM size. You can extend the role and maybe add other constraints and change
<code>max_connections</code> to you specific needs. An example playbook could look like:</p>
<div class="highlight"><pre><span></span>---
hosts: my_postgres
roles:
- postgres-config
vars:
- max_connection: 300
</pre></div>
<p>And run it:</p>
<div class="highlight"><pre><span></span>$ ansible-playbook playbook.yml
</pre></div>
<p>The complete role can be found in my <a href="https://github.com/tdi/postgres-config">github repo</a>.</p>HAProxy and 503 HTTP errors with AWS ELB as a backend2016-04-19T14:26:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2016-04-19:2016/04/19/haproxy-and-503-http-errors-with-aws-elb-as-a-backend/<p>Although, AWS provides load balancer service in the form of Elastic Load Balancer (ELB), a common
trick is to use HAProxy in the middle to provide SSL offloading, complex routing and better logging. <br />
In this scenario, a public ELB is the frontier of all the traffic, HAProxy farm in the middle is
managed by an Auto Scaling Group, and one (or more) internal backend ELBs stay in front of Web farm. </p>
<p><img alt="haproxy" src="//tdi.github.io/images/haproxy.png" /></p>
<p>I think that <a href="http://www.haproxy.org/">HAProxy</a> does not need any introductions here. It is highly
scalable and reliable piece of software. There is however a small caveat when you use it with domain
names and not IP addresses. To speed up things, HAProxy resolves all the domain named during startup (during config file parsing in fact). Hence, when
the IP of a domain changes, you end up with a lot of 503s (Service Unavailable). </p>
<p>Why is this important ? In AWS, ELB's IP can change over time, so it is recommended to use ELB's domain name.
Now, when you use this domain name in HAProxy's backend, you can end up with 503s. ELB IPs do not
change so often but still you would not want any downtimes. </p>
<p>The solution is to configure runtime resolvers in HAProxy and use them in the backend
<a href="http://blog.haproxy.com/2015/10/14/whats-new-in-haproxy-1-6/">(unforntunatelly this works only in HAProxy 1.6)</a>:</p>
<div class="highlight"><pre><span></span> :::haproxy
resolvers myresolver
nameserver dns1 10.10.10.10:53
resolve_retries 30
timeout retry 1s
hold valid 10s
backend mybackend
server myelb-internal.123456.eu-west-1.elb.amazonaws.com check resolvers myresolver
</pre></div>
<p>Now HAProxy will check the domain at runtime, no more 503s.</p>XWiki and slashes in URI2016-04-18T13:54:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2016-04-18:2016/04/18/xwiki-and-slashes-in-uri/<p><a href="http://www.xwiki.org/xwiki/bin/view/Main/WebHome">XWiki</a> is a great open source Atlassian Confluence replacement (some argue it is better, I leave it
to your assessment). We use XWiki a lot at Tenesys to document internal projects, and create
documentation of clients' platforms. We run XWiki in Tomcat application server, behind nginx proxy. </p>
<p>We use great XWiki's plugin, called FAQ, which can be used to create, well FAQs. The problem we had
was that sometimes people (me especially) created FAQ entries with a <code>/</code> in the name, which resulted
in XWiki creating a slug with <code>/</code> character, which is used to delimit page hierarchy in XWIki.
Basically, you wanted to write <code>How to install Debian/Ubuntu package</code> and you ended up with two
pages: <code>How to install Debian</code> and a subpage <code>Ubuntu package</code>. You can't easily delete the 'slashed'
FAQ page because by default the last one is deleted only. </p>
<p>The solution to this problems is twofold. First of all, you need to tell Tomcat to allow passing
encoded slash (<code>%2F</code>) oto XWiki.
Add to <code>-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true</code> to <code>CATALINA_OPTS</code>. You can
either do it via <code>catalina.sh</code> or <code>catalina.opts</code>.</p>
<p>Second of all, you need to make sure that your nginx proxy directive is bare, i.e. does not contain URI part, see relevant <a href="https://stackoverflow.com/questions/20496963/avoid-nginx-decoding-query-parameters-on-proxy-pass-equivalent-to-allowencodeds">stack question here</a>. Basically you want your <code>proxy_pass</code> to look like that:</p>
<div class="highlight"><pre><span></span><span class="k">location</span> <span class="s">/</span> <span class="p">{</span>
<span class="kn">proxy_pass</span> <span class="s">http://backend</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<p>... not like that.</p>
<div class="highlight"><pre><span></span><span class="k">location</span> <span class="s">/</span> <span class="p">{</span>
<span class="kn">proxy_pass</span> <span class="s">http://backend/xwiki</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<p>I spent quite a lot of time before I discovered that nginx caveat. Hope it helps somebody too. </p>Delete until signature in vim2015-09-29T16:13:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2015-09-29:2015/09/29/delete-until-signature-in-vim/<p>It has been bugging me for a while. When responding to an email, you often want to delete all the
content (or part of the previous content) until the end of the email's body. However it would be
nice to leave your signature in place. For that I came up with this nifty little vim trick:</p>
<div class="highlight"><pre><span></span><span class="nb">nnoremap</span> <span class="p"><</span><span class="k">silent</span><span class="p">></span> <span class="p"><</span>leader<span class="p">></span><span class="k">gr</span> <span class="p"><</span>Esc<span class="p">></span><span class="k">d</span>/<span class="p">--</span>\_.*Dariusz<span class="p"><</span>CR<span class="p">></span>:<span class="k">nohl</span><span class="p"><</span>CR<span class="p">></span>O
</pre></div>
<p>Assuming that your signature starts with <code>--</code> and the following line starts with your name (in my
case it is Dariusz), this will delete all the content from the current line until the signature.
Then it will remove search highlighting, and finally move one line up.</p>debrfstats software for RFS statistics2014-09-23T11:35:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2014-09-23:2014/09/23/debrfstats-software-for-rfs-statistics/<p><a href="//tdi.github.io/2014/09/21/statistics-of-rfs-bugs-and-sponsoring-process/">Last time</a> I told that I would release software I used to make
RFS stats plots. You can find it in my github repo -
<a href="https://github.com/tdi/debrfstats">github.com/tdi/debrfstats</a>. </p>
<p>The software contains small class to get data needed to generate plots, as well as for doing
some simple bug analysis. The software also contains an R script to make plots from a CSV file. For
now debrfstats uses SOAP interface to Debbugs but I am now working on adding a UDD data source.</p>
<p>The software is written in Python 2 (SOAPpy does not come in 3 flavour), some usage examples are in
the <code>main.py</code> file in the repository. </p>
<p>If you have any questions or wishes for debrfstats do not hesitate to contact me. </p>statistics of RFS bugs and sponsoring process2014-09-21T16:21:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2014-09-21:2014/09/21/statistics-of-rfs-bugs-and-sponsoring-process/<p>For some days I have been working on statistics of the sponsoring process in Debian. I find
this to be one of the most important things that Debian has to attract and enable new contributions.
It is important to know how this process works, whether we need more sponsors, how effective is the
sponsoring and what are the timings connected to it. </p>
<h1>How I did this ?</h1>
<p>I have used Debbugs SOAP interface to get all bugs that are filed against <code>sponsorship-requests</code>
pseudo package. SOAP gives a little bit of overhead because it needs to download a complete list of
bugs for the <code>sponsorship-requests</code> package, and then process them according to given date ranges.
The same information can be easily extracted from the UDD database in the future, it will be faster
because SQL is better when working with date ranges than python obviously. </p>
<p>The most problematic part was getting the "real done date" of a particular bug, and frankly most of
my time I have spent on writing a rather dirty and complicated script. The script gets a log for a
particular bug number and returns a "real done date". I have published a proof of concept in a
<a href="//tdi.github.io/2014/09/19/getting-real-done-date-from-debian-bts/">previous post.</a>.</p>
<h1>What I measured ?</h1>
<p>RFSs is a queue, and in every queue one is interested in a mean time to get processed. In this case I
called the metric global MTTGS (mean time to get sponsored). This is a metric that gives the overall
performance insight in RFS queue. Time to get sponsored (TTGS) for a bug is a number of days that
passed between filing an RFS bug and closing it (bug was sponsored). Mean time to get sponsored is calculated as
a sum of TTGSs of all bugs divided by number of bugs (in a given period of time). Global MTTGS is
MTTGS calculated for a period of time 2012-1-1 until <code>today()</code>. </p>
<p>Besides MTTGS I have also measured typical bug related metrics:</p>
<ul>
<li>number of bugs closed in a given day,</li>
<li>number of bugs opened in a given day,</li>
<li>number of bugs with status open in a given day,</li>
<li>number of bugs with status closed in a given day.</li>
</ul>
<h1>Plots and graphs</h1>
<p>Below is a plot of global MTTGS vs. time (click for a larger image).</p>
<p><a href="//tdi.github.io/images/mttgs.png"><img alt="mttgs plot" src="//tdi.github.io/images/mttgs_small.png" /></a></p>
<p>As you can see, the trend is roughly exponential and MTTGS tends to settle around 60 days at the end of the
year 2013. This does not mean that your package will wait 60 days on average nowadays to get
sponsored. I remind that this is a global MTTGS, so even if the MTTGS of last month was very low, the global MTTGS would decrease
just slightly. It gives, however, a good glance in performance of the process. Even that more
packages are filed for sponsoring (see next graphs) now, than in the beginning of the epoch, the
sponsoring rate is high enough to flatten the global MTTGS, and with time maybe decrease it. </p>
<p>The image below (click for a larger one) shows how many bugs reside in a queue with status open or
closed (calculated for each day). For closed we have an almost linear function, so each day more or less the same amount of
bugs are closed and they increase the pool of bugs with status closed. For bugs with status open the
interesting part begins around May 2012 after the system is saturated or gets popular. It can be interpreted
as a plot of how many bugs reside in the queue, the important part is that it is stable and does not
show clear increasing trend. </p>
<p><a href="//tdi.github.io/images/open_done.png"><img alt="open done plot" src="//tdi.github.io/images/open_done_small.png" /></a></p>
<p>The last plot shows arrival and departure rate of bugs from RFS queue, i.e. how many bugs are opened
and closed each day. The interesting part here are the maxima. Let's look at them.</p>
<p><a href="//tdi.github.io/images/opened_closed.png"><img alt="opened closed plot" src="//tdi.github.io/images/opened_closed_small.png" /></a></p>
<p>Maximal number of opened bugs (21) was on 2012-05-06. As it appears it was a bunch upload of RFSs
for <code>tryton-modules-*.</code>.</p>
<div class="highlight"><pre><span></span> 706953 RFS: tryton-modules-account-stock-anglo-saxon/2.8.0-1
706954 RFS: tryton-modules-purchase-shipment-cost/2.8.0-1
706948 RFS: tryton-modules-production/2.8.0-1
706969 RFS: tryton-modules-account-fr/2.8.0-1
706946 RFS: tryton-modules-project-invoice/2.8.0-1
706950 RFS: tryton-modules-stock-supply-production/2.8.0-1
706942 RFS: tryton-modules-product-attribute/2.8.0-1
706957 RFS: tryton-modules-stock-lot/2.8.0-1
706958 RFS: tryton-modules-carrier-weight/2.8.0-1
706941 RFS: tryton-modules-stock-supply-forecast/2.8.0-1
706955 RFS: tryton-modules-product-measurements/2.8.0-1
706952 RFS: tryton-modules-carrier-percentage/2.8.0-1
706949 RFS: tryton-modules-account-asset/2.8.0-1
706904 RFS: chinese-checkers/0.4-1
706944 RFS: tryton-modules-stock-split/2.8.0-1
706981 RFS: distcc/3.1-6
706945 RFS: tryton-modules-sale-supply/2.8.0-1
706959 RFS: tryton-modules-carrier/2.8.0-1
706951 RFS: tryton-modules-sale-shipment-cost/2.8.0-1
706943 RFS: tryton-modules-account-stock-continental/2.8.0-1
706956 RFS: tryton-modules-sale-supply-drop-shipment/2.8.0-1
</pre></div>
<p>Maximum number of closed bugs (18) was on 2013-09-24, and as you probably guessed right also tryton modules had impact on that. </p>
<div class="highlight"><pre><span></span> 706953 RFS: tryton-modules-account-stock-anglo-saxon/2.8.0-1
706954 RFS: tryton-modules-purchase-shipment-cost/2.8.0-1
706948 RFS: tryton-modules-production/2.8.0-1
706969 RFS: tryton-modules-account-fr/2.8.0-1
706946 RFS: tryton-modules-project-invoice/2.8.0-1
706950 RFS: tryton-modules-stock-supply-production/2.8.0-1
706942 RFS: tryton-modules-product-attribute/2.8.0-1
706958 RFS: tryton-modules-carrier-weight/2.8.0-1
706941 RFS: tryton-modules-stock-supply-forecast/2.8.0-1
706955 RFS: tryton-modules-product-measurements/2.8.0-1
706952 RFS: tryton-modules-carrier-percentage/2.8.0-1
706949 RFS: tryton-modules-account-asset/2.8.0-1
706944 RFS: tryton-modules-stock-split/2.8.0-1
706959 RFS: tryton-modules-carrier/2.8.0-1
723991 RFS: mapserver/6.4.0-2
706951 RFS: tryton-modules-sale-shipment-cost/2.8.0-1
706943 RFS: tryton-modules-account-stock-continental/2.8.0-1
706956 RFS: tryton-modules-sale-supply-drop-shipment/2.8.0-1
</pre></div>
<h1>The software</h1>
<p>Most of the software was written in Python. Graphs were generated in R. After a code cleanup I will
publish a complete solution on my github account, free to use by everybody. If you would like to see
another statistics, please let me know, I can create them if the data provides sufficient
information. </p>getting real "done date" of a bug from Debian BTS2014-09-19T09:17:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2014-09-19:2014/09/19/getting-real-done-date-from-debian-bts/<p>As I wrote in my last <a href="//tdi.github.io/2014/09/18/rfs-health-in-debian/">post</a> currently, SOAP
interface, nor Ultimate Debian Database do not provide a date when a given bug was closed (done
date). It is quite hard to calculate statistics on a bug tracker when you do not know when a bug was
closed (!!). </p>
<p>Done date of bug can be found in its log. The log itself can be downloaded by SOAP method
<code>get_bug_log</code> but the processing of it is quite complicated. The same comes to web scrapping of a
BTS's web interface. Fortunatelly the web interface gives a possibility to download a log in an mbox
format. </p>
<p>Below is a script that extracts the done date of a bug from its log in mbox format. It uses requests
to download the mbox and caches the result in <code>~/.cache/rfs_bugs</code>, which you need to create. It performs
different checks:</p>
<ol>
<li>Check existence of a header e.g. <code>Received: (at 657783-done) by bugs.debian.org; 29 Jan 2012
13:27:42 +0000</code></li>
<li>Check for header <code>CC: NUMBER-close|done</code></li>
<li>Check for header <code>TO: NUMBER-close|done</code></li>
<li>Check for <code>Close: NUMBER</code> in body.</li>
</ol>
<p>The code is below:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="kn">import</span> <span class="nn">mailbox</span>
<span class="kn">import</span> <span class="nn">re</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">tempfile</span>
<span class="k">def</span> <span class="nf">get_done_date</span><span class="p">(</span><span class="n">bug_num</span><span class="p">):</span>
<span class="n">CACHE_DIR</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">expanduser</span><span class="p">(</span><span class="s2">"~"</span><span class="p">)</span> <span class="o">+</span> <span class="s2">"/.cache/rfs_bugs/"</span>
<span class="k">def</span> <span class="nf">get_from_cache</span><span class="p">():</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="s2">"{}{}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">CACHE_DIR</span><span class="p">,</span> <span class="n">bug_num</span><span class="p">)):</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"{}{}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">CACHE_DIR</span><span class="p">,</span> <span class="n">bug_num</span><span class="p">))</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="k">return</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">readlines</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">rstrip</span><span class="p">(),</span> <span class="s2">"%Y-%m-</span><span class="si">%d</span><span class="s2">"</span><span class="p">)</span><span class="o">.</span><span class="n">date</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="n">done_date</span> <span class="o">=</span> <span class="n">get_from_cache</span><span class="p">()</span>
<span class="k">if</span> <span class="n">done_date</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
<span class="k">return</span> <span class="n">done_date</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"https://bugs.debian.org/cgi-bin/bugreport.cgi?mbox=yes;bug={};mboxstatus=yes"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_num</span><span class="p">))</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">try_header</span><span class="p">(</span><span class="n">r</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">try_cc</span><span class="p">(</span><span class="n">r</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">try_body</span><span class="p">(</span><span class="n">r</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"{}{}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">CACHE_DIR</span><span class="p">,</span> <span class="n">bug_num</span><span class="p">),</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s2">"{}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">d</span><span class="o">.</span><span class="n">date</span><span class="p">()))</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">return</span> <span class="n">d</span><span class="o">.</span><span class="n">date</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">try_body</span><span class="p">(</span><span class="n">text</span><span class="p">):</span>
<span class="n">reg</span> <span class="o">=</span> <span class="s2">"\(at\s.+\)\s+by\sbugs\.debian\.org;\s(\d{1,2}\s\w\w\w\s\d\d\d\d)"</span>
<span class="n">handle</span><span class="p">,</span> <span class="n">name</span> <span class="o">=</span> <span class="n">tempfile</span><span class="o">.</span><span class="n">mkstemp</span><span class="p">()</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">text</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'latin-1'</span><span class="p">))</span>
<span class="n">mbox</span> <span class="o">=</span> <span class="n">mailbox</span><span class="o">.</span><span class="n">mbox</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">mbox</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">if</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">is_multipart</span><span class="p">():</span>
<span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">get_payload</span><span class="p">():</span>
<span class="k">if</span> <span class="s2">"close"</span> <span class="ow">in</span> <span class="nb">str</span><span class="p">(</span><span class="n">m</span><span class="p">)</span> <span class="ow">or</span> <span class="s2">"done"</span> <span class="ow">in</span> <span class="nb">str</span><span class="p">(</span><span class="n">m</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">reg</span><span class="p">,</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">][</span><span class="s1">'Received'</span><span class="p">])</span>
<span class="k">return</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="s2">"</span><span class="si">%d</span><span class="s2"> %b %Y"</span><span class="p">)</span>
<span class="k">except</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">if</span> <span class="s2">"close"</span> <span class="ow">in</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">get_payload</span><span class="p">()</span> <span class="ow">or</span> <span class="s2">"done"</span> <span class="ow">in</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">get_payload</span><span class="p">():</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">reg</span><span class="p">,</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">][</span><span class="s1">'Received'</span><span class="p">])</span>
<span class="k">return</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="s2">"</span><span class="si">%d</span><span class="s2"> %b %Y"</span><span class="p">)</span>
<span class="k">except</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">def</span> <span class="nf">try_header</span><span class="p">(</span><span class="n">text</span><span class="p">):</span>
<span class="n">reg</span> <span class="o">=</span> <span class="s2">"Received:\s\(at\s\d\d\d\d\d\d-(close|done)\)\s+by.+"</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">reg</span><span class="p">,</span> <span class="n">r</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
<span class="n">line</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="n">reg2</span> <span class="o">=</span> <span class="s2">"\d{1,2}\s\w\w\w\s\d\d\d\d"</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">reg2</span><span class="p">,</span> <span class="n">line</span><span class="p">)</span>
<span class="n">d</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> <span class="s2">"</span><span class="si">%d</span><span class="s2"> %b %Y"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">d</span>
<span class="k">except</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">def</span> <span class="nf">try_cc</span><span class="p">(</span><span class="n">text</span><span class="p">):</span>
<span class="n">reg</span> <span class="o">=</span> <span class="s2">"\(at\s.+\)\s+by\sbugs\.debian\.org;\s(\d{1,2}\s\w\w\w\s\d\d\d\d)"</span>
<span class="n">handle</span><span class="p">,</span> <span class="n">name</span> <span class="o">=</span> <span class="n">tempfile</span><span class="o">.</span><span class="n">mkstemp</span><span class="p">()</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">text</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'latin-1'</span><span class="p">))</span>
<span class="n">mbox</span> <span class="o">=</span> <span class="n">mailbox</span><span class="o">.</span><span class="n">mbox</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">mbox</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">if</span> <span class="p">(</span><span class="s1">'CC'</span> <span class="ow">in</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="ow">and</span> <span class="s2">"done"</span> <span class="ow">in</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">][</span><span class="s1">'CC'</span><span class="p">])</span> <span class="ow">or</span> <span class="p">(</span><span class="s1">'To'</span> <span class="ow">in</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="ow">and</span> <span class="s2">"done"</span> <span class="ow">in</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">][</span><span class="s1">'To'</span><span class="p">]):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">reg</span><span class="p">,</span> <span class="n">i</span><span class="p">[</span><span class="mi">1</span><span class="p">][</span><span class="s1">'Received'</span><span class="p">])</span>
<span class="k">return</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">group</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="s2">"</span><span class="si">%d</span><span class="s2"> %b %Y"</span><span class="p">)</span>
<span class="k">except</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">None</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="k">print</span> <span class="n">get_done_date</span><span class="p">(</span><span class="mi">752210</span><span class="p">)</span>
</pre></div>
<p>PS: I hope that the script will be not needed in the near future, as Don Armstrong plans a new BTS
database, a <a href="http://meetings-archive.debian.net/pub/debian-meetings/2014/debconf14/webm/bugsdebianorg_Database_Ho.webm">Debconf14 video is here</a>.</p>RFS health in Debian2014-09-18T10:50:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2014-09-18:2014/09/18/rfs-health-in-debian/<p>I am working on a small project to create WNPP like statistics for open RFS bugs. I think this could
improve a little bit effectiveness of sponsoring new packages by giving insight into bugs that are on
their way to being starved (i.e. not ever sponsored, or rotting in a queue). </p>
<p>The script attached in this post is written in Python and uses <a href="https://wiki.debian.org/DebbugsSoapInterface">Debbugs SOAP
interface</a> to get currently open RFS bugs and
calculates their dust and age. </p>
<p>The dust factor is calculated as an absolute value of a difference between bugs's <code>age</code> and
<code>log_modified</code>. </p>
<p>Later I would like to create fully blown stats for an RFS queue, taking into account
the whole history (i.e. 2012-1-1 until now), and check its health, calculate <code>MTTGS</code> (mean time to get sponsored). </p>
<p>The list looks more or less like this:</p>
<div class="highlight"><pre><span></span>Age Dust Number Title
37 0 757966 RFS: lutris/0.3.5-1 [ITP]
1 0 762015 RFS: s3fs-fuse/1.78-1 [ITP #601789] -- FUSE-based file system backed by Amazon S3
81 0 753110 RFS: mrrescue/1.02c-1 [ITP]
456 0 712787 RFS: distkeys/1.0-1 [ITP] -- distribute SSH keys
120 1 748878 RFS: mwc/1.7.2-1 [ITP] -- Powerful website-tracking tool
1 1 762012 RFS: fadecut/0.1.4-1
3 1 761687 RFS: abraca/0.8.0+dfsg-1 -- Simple and powerful graphical client for XMMS2
35 2 758163 RFS: kcm-ufw/0.4.3-1 ITP
3 2 761636 RFS: raceintospace/1.1+dfsg1-1 [ITP]
....
....
</pre></div>
<p>The script <code>rfs_health.py</code> can be found below, it uses <code>SOAPpy</code>
(only python <3 unfortunately).</p>
<div class="highlight"><pre><span></span><span class="ch">#!/usr/bin/python</span>
<span class="kn">import</span> <span class="nn">SOAPpy</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">date</span><span class="p">,</span> <span class="n">timedelta</span><span class="p">,</span> <span class="n">datetime</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">'http://bugs.debian.org/cgi-bin/soap.cgi'</span>
<span class="n">namespace</span> <span class="o">=</span> <span class="s1">'Debbugs/SOAP'</span>
<span class="n">server</span> <span class="o">=</span> <span class="n">SOAPpy</span><span class="o">.</span><span class="n">SOAPProxy</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">namespace</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">RFS</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_obj</span> <span class="o">=</span> <span class="n">obj</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_last_modified</span> <span class="o">=</span> <span class="n">date</span><span class="o">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="n">log_modified</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_date</span> <span class="o">=</span> <span class="n">date</span><span class="o">.</span><span class="n">fromtimestamp</span><span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="n">date</span><span class="p">)</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_obj</span><span class="o">.</span><span class="n">pending</span> <span class="o">!=</span> <span class="s1">'done'</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_pending</span> <span class="o">=</span> <span class="s2">"pending"</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_dust</span> <span class="o">=</span> <span class="nb">abs</span><span class="p">(</span><span class="n">date</span><span class="o">.</span><span class="n">today</span><span class="p">()</span> <span class="o">-</span> <span class="bp">self</span><span class="o">.</span><span class="n">_last_modified</span><span class="p">)</span><span class="o">.</span><span class="n">days</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_pending</span> <span class="o">=</span> <span class="s2">"done"</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_dust</span> <span class="o">=</span> <span class="nb">abs</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_date</span> <span class="o">-</span> <span class="bp">self</span><span class="o">.</span><span class="n">_last_modified</span><span class="p">)</span><span class="o">.</span><span class="n">days</span>
<span class="n">today</span> <span class="o">=</span> <span class="n">date</span><span class="o">.</span><span class="n">today</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_age</span> <span class="o">=</span> <span class="nb">abs</span><span class="p">(</span><span class="n">today</span> <span class="o">-</span> <span class="bp">self</span><span class="o">.</span><span class="n">_date</span><span class="p">)</span><span class="o">.</span><span class="n">days</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">status</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_pending</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">date</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_date</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">last_modified</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_last_modified</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">subject</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_obj</span><span class="o">.</span><span class="n">subject</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">bug_number</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_obj</span><span class="o">.</span><span class="n">bug_num</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">age</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_age</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">dust</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_dust</span>
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"{} subject: {} age:{} dust:{}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_obj</span><span class="o">.</span><span class="n">bug_num</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_obj</span><span class="o">.</span><span class="n">subject</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_age</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_dust</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="n">bugi</span> <span class="o">=</span> <span class="n">server</span><span class="o">.</span><span class="n">get_bugs</span><span class="p">(</span><span class="s2">"package"</span><span class="p">,</span> <span class="s2">"sponsorship-requests"</span><span class="p">,</span> <span class="s2">"status"</span><span class="p">,</span> <span class="s2">"open"</span><span class="p">)</span>
<span class="n">buglist</span> <span class="o">=</span> <span class="p">[</span><span class="n">RFS</span><span class="p">(</span><span class="n">b</span><span class="o">.</span><span class="n">value</span><span class="p">)</span> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">server</span><span class="o">.</span><span class="n">get_status</span><span class="p">(</span><span class="n">bugi</span><span class="p">)</span><span class="o">.</span><span class="n">item</span><span class="p">]</span>
<span class="n">buglist_sorted_by_dust</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">buglist</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">dust</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s2">"Age Dust Number Title"</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">buglist_sorted_by_dust</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="s2">"{:<4} {:<4} {:<7} {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="o">.</span><span class="n">age</span><span class="p">,</span> <span class="n">i</span><span class="o">.</span><span class="n">dust</span><span class="p">,</span> <span class="n">i</span><span class="o">.</span><span class="n">bug_number</span><span class="p">,</span> <span class="n">i</span><span class="o">.</span><span class="n">subject</span><span class="p">))</span>
</pre></div>forwarding messages with attachments in mutt2014-09-12T14:30:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2014-09-12:2014/09/12/forwarding-messages-with-attachments-in-mutt/<p>This is a pain for every mutt user. I do not know why this solution is so hard to find.
Just add these two lines to your <code>.muttrc</code>.</p>
<div class="highlight"><pre><span></span>set mime_forward
set mime_forward_rest=yes
</pre></div>
<p>This will forward an email with all the attachments, no scripts needed, no fancy tagging or
reediting. </p>profanity and libstrophe status in Debian2014-09-12T09:10:00+02:00Dariusz Dwornikowskitag:tdi.github.io,2014-09-12:2014/09/12/profanity-and-libstrophe-status-in-debian/<p><a href="http://www.profanity.im">profanity</a> is a great console based XMPP client written in ncurses and C
by James Booth. The code has a great quality, upstream is super collaborative, and willing, so packaging should be pretty straightforward. This
post will show that this was not the case here. </p>
<p><img alt="profanity" src="http://www.profanity.im/images/prof-1.png" /></p>
<p>First obstacle was that profanity depended on <a href="http://strophe.im">libstrophe</a>, an XMPP library, which
was not in Debian. As it occurred libstrophe's upstream was not responsive, so any changes that were
needed to prepare libstrophe for high quality packaging could not be met. </p>
<ol>
<li>First of all libstrophe's build system (automake and friends) built only a static library. </li>
<li>The second problem was that libstrophe did not tag releases on github, this was needed to make Debian watch file work. </li>
<li>A third, smaller problem was the presence of <code>debian/</code> directory in upstream's source. It can be neglected
most of the time, since you can tell <code>git-import-orig</code> to delete it. </li>
</ol>
<p>To solve those 3 problems I created a pull request fixing the
build system to build also a shared library, deleting <code>debian/</code> directory and politely asking for
tagging releases. You can see my pull request <a href="https://github.com/strophe/libstrophe/pull/20">here</a> dated on April 26th.
There was no answer for the libstrophe's upstream but I has some support from profanity's developers
and other users wanting to make those changes. Finally metajack (libstrophe upstream) gave us right
to the repo and we could merge the pull request on August 6th. The lesson learned - be patient and
know autotools (a great tutorial <a href="https://www.lrde.epita.fr/~adl/autotools.html">is here</a>). </p>
<p>With profanity there were less changes to do. The most important one was that it linked to OpenSSL
and due to the license incompatibility with GPL it could not go into Debian. Fortunately upstream
added the OpenSSL exception, and profanity could be finally packaged. </p>
<p>Now both profanity and libstrophe are in <a href="https://ftp-master.debian.org/new.html">NEW queue</a> and hopefully they will be accepted by ftp
masters. When they are, there is plenty to do with them in the future, upstream closed some bugs,
new upstream versions are tagged. </p>VIM PEPA syntax2013-02-18T21:22:41+01:00Dariusz Dwornikowskitag:tdi.github.io,2013-02-18:2013/02/18/vim-pepa-syntax/<p>For everyone interested in PEPA (Performance Evaluation Process Algbra) I created a small
syntax plugin that will colorize syntax of any PEPA model in VIM. The plugin is hosted on github
<a class="reference external" href="https://github.com/tdi/vim-pepa-syntax">here</a>.</p>
<p>Here is how it looks:</p>
<img alt="pepa vim plugin in action" class="align-center" src="//tdi.github.io/images/pepa-vim.png" />
Highlight lines over 792013-02-02T11:09:54+01:00Dariusz Dwornikowskitag:tdi.github.io,2013-02-02:2013/02/02/highlight-lines-over-79/<p>If you want to stick to PEP8 speccification of Python syntax, you should stick to 79 line length.
It is very easy to forget to follow that rule, fortunatelly vim can help you. There are many nice
solution to inform you when you go past 79 column. The most generic one is highlighting only the
80th column by setting <tt class="docutils literal">set cursorcolumn</tt> (or just <tt class="docutils literal">set cc</tt>). This will produce a vertical line
on the column according to your <tt class="docutils literal">textwidth</tt> variable. You can check this by doing <tt class="docutils literal">:set tw?</tt>.
If you want some better looking solution try the one I found on stackoverflow. You can adjust the color and linewidth to your
preferences.</p>
<div class="highlight"><pre><span></span><span class="nb">highlight</span> OverLength ctermbg<span class="p">=</span><span class="k">red</span> ctermfg<span class="p">=</span>white guibg<span class="p">=</span><span class="mh">#592929</span>
<span class="k">match</span> OverLength <span class="sr">/\%80v.\+/</span>
</pre></div>
Text above arrows in LaTeX2013-01-30T12:00:39+01:00Dariusz Dwornikowskitag:tdi.github.io,2013-01-30:2013/01/30/text-above-arrows-in-latex/<p>Sometimes you need to place text above arrows. Let's say you need to write transitions for a labelled transition system. In fact in LaTeX this is not so obvious, especially if you want to have text above and under the arrow. I know of two easy ways to do it.</p>
<div class="highlight"><pre><span></span><span class="sb">$$</span><span class="nb"> E </span><span class="nv">\xrightarrow</span><span class="nb">{</span><span class="nv">\alpha</span><span class="nb">} F </span><span class="s">$$</span>
</pre></div>
<p>The second method.</p>
<div class="highlight"><pre><span></span><span class="sb">$$</span><span class="nb"> </span><span class="nv">\mathop</span><span class="nb">{</span><span class="nv">\longrightarrow</span><span class="nb">}^_{</span><span class="nv">\alpha</span><span class="nb">} </span><span class="s">$$</span>
</pre></div>
Toggle cursorline and cursorcolumn function in VIM2013-01-30T10:59:09+01:00Dariusz Dwornikowskitag:tdi.github.io,2013-01-30:2013/01/30/toggle-cursorline-in-vim/<p>It is often very comfortable to see where your cursor in VIM is. To achieve that you can use
cursorcolumn and cursorline to highlight the row and the column in which you are currently present
with your cursor. Below is a function that can be placed in your <tt class="docutils literal">.vimrc</tt> to toggle such a
behaviour. It is then mapped to <tt class="docutils literal"><leader>cl</tt>, which effectively means that you need to punch <tt class="docutils literal">\cl</tt> to
make it work.</p>
<div class="highlight"><pre><span></span><span class="k">set</span> <span class="nb">cursorline</span>
<span class="k">set</span> <span class="nb">cursorcolumn</span>
<span class="k">fu</span><span class="p">!</span> ToggleCurline <span class="p">()</span>
<span class="k">if</span> &<span class="nb">cursorline</span> && &<span class="nb">cursorcolumn</span>
<span class="k">set</span> <span class="nb">nocursorline</span>
<span class="k">set</span> <span class="nb">nocursorcolumn</span>
<span class="k">else</span>
<span class="k">set</span> <span class="nb">cursorline</span>
<span class="k">set</span> <span class="nb">cursorcolumn</span>
<span class="k">endif</span>
<span class="k">endfunction</span>
map <span class="p"><</span><span class="k">silent</span><span class="p">><</span>leader<span class="p">></span><span class="k">cl</span> :<span class="k">call</span> ToggleCurline<span class="p">()<</span>CR<span class="p">></span>
</pre></div>