Question

#

What are the tools I use daily that I could contribute to?

Answer

#

General

#

• Visual Studio Code My text editor of choice when I don’t need an IDE. I’ve implemented a few plugins for VSC and use it daily to write personal notes as well as blog articles. I sometimes make use of its diff tool to merge changes from Sourcetree.
• Pycharm My IDE of choice when I write python. Very powerful, easy to get used to if you’ve used other Jetbrains IDE (I’ve used PHPStorm for more than 5 years). Extremely useful to run a specific unit test using pytest or to debug a complex issue by putting breakpoints and investigating the internal state of the program.
• Docker I use docker at work to containerize all of our dependencies so that it is somewhat easy to deploy what we develop in “any” environment, that is, an environment where docker (or similar, such as Kubernetes) is installed.
• Drone CI We use Drone CI at work to do continuous integration and I consider this tool to be an essential part of my daily work. When people push code to github and it fails on Drone CI, I can use this information to help them fix their issues. We also use it has part of our PR process to ensure that the PR passes all the expected tests so that we do not introduce faulty code into our master branch.
• Dependabot A few months ago I had introduced dependabot into my dependency management practices. It was highly useful to get automated PRs with updates to libraries we depended on. However, since the release of poetry 1.0.0, dependabot has not been able to update my python dependencies and has been left unused. I’ve created a PR which I hope will move this issue forward and get dependabot working again with poetry.
• Plotly I used to use the highcharts plotting library until someone at work introduced me to plotly. Plotly.js is open source software and released under the MIT license, which makes it an ideal library to use in personal as well as commercial software.

Python specific

#

• Pandas I do machine learning development for a living nowadays and I depend highly on pandas. I don’t think there’s a single work day that goes by in which I don’t whip out at least one pd.DataFrame.
• Scikit-learn Similar to my dependency on pandas, my dependency on Scikit-learn is on a daily basis. Unlike the DummyRegressor documentation suggests, I use it for real problems and it’s definitely useful!
• Dask / Distributed In order to scale both horizontally and vertically machine learning problems I’ve leaned on dask and distributed. Their use of delayed and Futures has made it simple to migrate simple for loops code into highly distributed tasks which can be monitored through a bokeh dashboard.
• pytest Who writes code without testing it? Pytest is the PHPUnit of python for me, an essential component that is used daily to ensure that code doesn’t regress more than it needs to.
• mypy Python typing system is pretty weak in my opinion. I miss using PHP typing system, as well as its visibility system. Mypy is similar to doing a code compilation pass and verifying that the types specified in a function signature are the types of the arguments given to that function. It is useful in order to detect mistakes in the arguments being passed to a function.
• isort I’m a tidy man. I like when my imports are ordered alphabetically. That’s what isort is there for.
• black I don’t particularly like discussing code style with others because everyone has their own quirks and creating a code style that everyone agrees on is as difficult as agreeing on whether tabs or spaces should be used. black is highly opinionated and doesn’t allow for much to be tweaked, while it also has a sensible style that sometimes can make you crazy.
• prospector Prospector allows you to run a variety of linters on your code, which is quite useful when you like your code to be as standard and pretty as I like it. Some of the tools also look for code complexity, which helps you identify nightmares before they’re in the master branch.
• poetry I’ve used pip, I’ve used pipenv, requirements.txt, setup.py, etc. I didn’t like the setup.py because since I’ve used composer (for PHP), I’ve always seen dependency management as something that shouldn’t require code to define. I didn’t like the requirements.txt/.lock variants because it was never clear how those were generated and if they were kept up to date together since you could use the requirements.txt as soft dependencies and requirements.lock as hard dependencies that you had to freeze yourself (which many people didn’t know about). pipenv Pipfile was alright, but adding dependencies seemed to take longer and longer, which wasn’t a pleasant experience, especially when the package you wanted to add didn’t want to play nice with the other packages. poetry was the closest experience I got to composer.

Question

#

Is a programmer commentary something that can provide value?

Answer

#

A few years ago I was learning how to drive. One of the techniques they teach in order to learn the basics of driving and help you reduce the amount of stress related to driving is to comment on what you are doing: I’m looking at this panel that says the maximum speed is 50 km/h, I am watching in my mirror, I am signaling that I want to move to the right lane, I’m checking my right dead angle, etc. The use of this technique is that it makes what you are doing in your head explicit by saying it out loud. There’s a similar technique called pointing and calling that is used by Japanese and Chinese railways drivers in order to keep focus and attention.

When we’re doing pair programming with someone else, we will often communicate with the other person what our current intent is and what we’re currently trying to write down. It may help us clarify our objectives or express clearly the various steps we’ll have to go through. In the case of pair programming, one of the benefits is that someone else is there to double-check what you are doing and what you’re about to do.

I’m the context where you are programming by yourself, you can still benefit from talking out loud about what you are doing. If you are willing, you can record yourself and even possibly use speech to text technologies to create a log of what you were doing at the time. This can be useful to recall what you were working on if you are interrupted at some point. It can also serve as a good way to get back context from a piece of code if you need to work on it again.

Writing down comments can also be useful, however there’s a lot of thinking that goes into programming that is only transient and if it were to be written as comments, it would make the code more difficult to consume.

References

#

• Pointing and calling

Question

#

How do checklists help to avoid mistakes?

Answer

#

Checklists are designed to ensure that the most important things, either it is an item or a step in a process, are not forgotten.

A process checklist will list all the steps that are critical to the completion of a process. It will make clear what steps need to be done and verified in what order. Checklists are a way to communicate with new team members what the existing team processes are and what is expected of them. As steps are completed, they are checked to indicate that the task was done or verified, which can serve as a progress indicator when a process takes a while to accomplish. What is also great about checklists is that it can be shared between teams as a way to share knowledge about processes.

If you often make mistakes in certain processes you do throughout the day, you can easily create a checklist that lists all the important things that you need to double-check when you do that process. You can then go through that list while completing the process or after completing it to ensure that all the important aspects of the process were correctly done.

One benefit of the checklist is that it also allows you to offload information that you might have to keep in your head at all times. A checklist for a process you do very infrequently can be an amazing time-saving tool since you do not need to refresh your memory on all the important steps of that process.

Anything that you can memorize, you should write down so that you can consult later and improve over time.

Question

#

How does one keep a library organized if people are moving books improperly at a certain rate?

Answer

#

In order to keep the library as ordered as possible, librarian should suggest that books that have been removed from the shelves be put onto cart or tables where those will be properly shelved again. Given that an individual may think they properly shelved a book, they might have actually introduced a slight error in the ordering of the books. If there is enough disorder created through this method, then it becomes more challenging for librarians to keep the books ordered.

We know from computer science that efficient sorting algorithms are of complexity O(n log n). For a nearly sorted initial order, insertion sort is considered to be the best as it will become close to O(n) complexity (while it is O(n^2) in the average/worst case).

Question

#

How do I make complex decisions?

Answer

#

When I have a decision that requires me to consider a lot of aspects, I’ve always documented my thoughts. I would first start by writing down all the things that come to mind in relation to this decision: things I want to consider, things I don’t want to forget, things that may be difficult to include as part of the decision, etc. I write as much as I can. Then I order the items I’ve listed by order of importance. What is it in all those ideas that I value the most?

When I’ve created a few alternative decisions, I evaluate each of the pros and cons. I use my list of importance to determine the solution that best aligns with those values. Oftentimes I will have an acceptable decision, but there will be parts of the decision that will need to be ironed out so that it is a good decision.

I always consider how important the decision is to determine how much time I will spend on that decision. There is no value in optimizing the decisions that have a negligible impact on you, but important decisions can definitely have a long term impact on your life. Note that I didn’t say that complex decisions were important decisions. Sometimes complex decisions are not important at all. In that case, it is not worth thinking about this decision for too long.

Question

#

Should I solve my problem with AI?

Answer

#

First start with a non-AI solution, then look into AI if what you have is unsatisfactory.

In this day and age, we want to make AI solve all kinds of problems, even those that don’t require AI.

When you build a product, you generally have a problem you are trying to solve. With AI, companies are looking at existing problems and trying to find ways to turn them into AI problems. It is not a problem in itself to attempt the exercise, but it is a mistake to implement an AI solution where a non-AI solution would’ve been more than adequate. There is still a lot of work that needs to be done in the field of automation that does not require AI, but simpler statistical approaches.

A lot of research in the field of machine learning and deep learning like to point to Occam’s razor in order to create simple models, but they sometimes seem to forget to apply the same principle to the whole solution, that is, do I need AI for this or would something simpler be as good?

A problem with the current wave of AI companies is their relation to AI itself. They corner themselves into doing AI only, while AI still highly relies on programming and IT, which are still highly technical but a lot less glamorous. It is definitely exciting to sell products that have AI in them, but starting with the tool and not the problem that needs to be solved is similar to trying to find all the problems a hammer can solve instead of knowing that you need a hammer when you want to nail something.

Question

#

Is there value in writing everything down?

Answer

#

I had this question when I started my research journey towards AGI. At one point I realized I wouldn’t be able to hold everything in my brain and that I needed to write down what I was thinking about so that I would be able to come back later, re-read what I wrote and continue making progress on what I was working on, whether it had been a day, a few weeks, or many months.

There is definite value in writing everything down. It frees your mind, letting you delegate the task of remembering to the computer. A thought I might have had more than five years ago can still be accessible to me if I wrote it down, but it’s unlikely I’ll remember exactly what I was thinking five years ago by “thinking really hard”. Nowadays I have trouble even remembering what I was working on a few hours prior due to the high volume of interruption I face in my work. Writing things down allows me to keep a record of what I’m doing.

One of the main concerns of writing everything down is that it can be seen as a waste of time. We often write and never read what we wrote again. As I write more and more, it becomes less and less likely that I will read what I’ve written in the past because the volume of content increases requiring more time to read. For the same amount of time spent reading, I cover the same constant amount of content, but the amount of uncovered content keeps growing, assuming I review prior content more than once.

I don’t view writing that is not read again to be wasteful. Sometimes writing down our thoughts allows us to figure out an issue that is blocking us. Sometimes expressing what we feel makes us realize that the feelings themselves are very hard to explain. Sometimes we just need a buffer where we can jot things down and get rid of it once we’re done.

Writing (and sharing) has the benefit that your thoughts can be reused by others. They can also be reused by your future self if you end up facing the same situation again.

References

#

• How often should I review my personal notes?
• Research methodology and organization

Question

#

What is the most complex sequence that can be made from a n-long binary string?

Answer

#

0….0 and 1….1 are easily compressible, thus what is the most unpredictable sequence?

Let’s proceed by induction.

1-long binary string

#

0 and 1 have a 50% to be selected in a 1-long binary string, thus they are both the most complex examples of a 1-long binary string.

2-long binary string

#

With two bits, we introduce two patterns: repeating and alternating. Repeating means that the bits are repeated for the complete string. Alternating means that the bits are alternating between 0 and 1 given a certain periodicity.

From our initial observation we suggested that the repeating pattern was likely to be the simplest because it consists of defining the symbol (0 or 1) that is repeated and the number of repetitions. We can say that this function has two parameters: length and symbol.

The same can be said about the alternating pattern. If you provide the initial symbol and the length of the pattern, you’ve defined its parameters. We can say that this function also has two parameters: length and starting symbol.

Using those two descriptions, we can observe that the alternating pattern is in fact a special case of the repeating pattern, where the symbol is 2 bits instead of one.

00 = repeating(2, 0) 01 = alternating(2, 0) = repeating(1, 01) 10 = alternating(2, 1) = repeating(1, 10) 11 = repeating(2, 1)

3-long binary string

#

001, 011, 100, 110 can be described as alternating and repeating, which is more complex than the previous examples as it requires the composition of two operations.

4 complex examples (001, 011, 100, 110) have been introduced at this stage.

4-long binary string

#

Here we first observe the presence of the alternating pattern with a periodicity of 2. We also observe the first case of a mirror pattern with 0110 and 1001. It could be claimed that 010 and 101 is the uneven equivalent of the mirror pattern for 3-long binary strings. The remaining “complex” examples are again displaying alternating and repeating patterns at the lower level.

One observation is that the complex patterns appear to be the variants where 1 bit is the opposite of the remaining bits, the only special case so far being for the 3-long binary string where the pattern is considered alternating. Instead of considering it a complex pattern, it could be described as the operation “fill X bits with 0/1, then flip by Y”.

From those few inductive iterations, we can see that the first and last string are always the repeating pattern. We also observe that the first half of the strings exhibit the same pattern of the second half of the strings, which is expected due to the binary nature of the string.

At this point there is still not enough information to be able to derive an answer to the original question, given that the strings can be described using 4 patterns: repeating, alternating, mirror, and fill & flip.

8 complex examples are found at this stage.

5-long binary string

#

The second half of this table was not written down due to the previous observation that the patterns are the same, simply inverted.

At this stage, the complex patterns that emerge are difficult to express using the prior language. Composition needs to be used, e.g. 00101 can be described as 00 then 101.

14 complex examples are found at this stage.

At this point in my study, I would summarize as follows:

• Repeating, alternating and fill & flip are low complexity. For a n-long binary string, this implies 2 repeating patterns, 2n fill & flip patterns, 2n alternating patterns of periodicity=1.
• Complexity appears to emerge from alternating patterns with varying periodicity. As we’re working with longer and longer strings, the periodic patterns will start to become periodic themselves (I believe this is leading to fractal).
• The fill & flip pattern appears to be able to cover most of the complex cases that appear as the length of the binary string increases.
• Some of the patterns can be expressed as a few alternatives, for example 01110 as alternating, periodicity=1,3,1, but also as 0 and fill & flip (or even an “odd” mirror). My intuition would lead me to think that those cases that can be expressed in multiple ways are less complex.
• It wasn’t done during this analysis, but it may be possible to apply operations overlayed on one another, such as two alternating patterns, one with a periodicity of 2 and one with a periodicity of 3, and use logic operators such as AND/OR/XOR (and their negative variants) to recreate the pattern
• The sequence of complex examples seems to be as follows: 0, 4, 8, 14, …
• It would be interesting to look at those strings from a logic circuit point of view. For example, 00000 can be produced from a single signal (0) routed to 5 exits, 00001 would be a single signal (0) routed to 4 exits and to one not gate which is then routed to an exit. Assuming the input signal is always 0, producing 11111 would require 1 not gate.

Question

#

What can cause us to learn inefficiently?

Answer

#

There are many reasons which can cause someone to learn at a slower pace than what should be possible for them. This can be caused by many factors, amongst them:

• Learning the same material under different wordings: You are going through material which you’ve already read, thus this “new” material is not providing you with any fresh information.
• Learning outdated material: You are learning material that has already been replaced with better material. It may be useful to learn outdated material for the historical purpose, but you should always aim to know what is the latest material.
• Learning incorrect material: You are learning material that is effectively not providing you with the information you want to learn. It may appear like you are learning something, but learning the wrong facts is a waste of your time.
• Learning from a source that makes the material difficult to understand: Learning efficiently is about learning from the best sources. If the material you have makes it difficult to understand what you are trying to learn, you should look for content elsewhere.
• Learning material for which we do not know the prerequisites: Learning a topic when we do not have the prerequisite knowledge will generally lead to confusion and make it difficult, if not impossible to learn.
• Learning from multiple sources with contradictory information: Learning from sources that disagree on facts makes it difficult to learn because you have to either accept that both answers are true, which can be complicated when they contradict each other, or that you need to establish which answer is the correct one, which requires additional effort.
• Learning while not being completely focused on the task: When learning, it is imperative to dedicate our focus to the task. Not doing so is likely to lead to information falling through the cracks of focus, leading to gaps in understanding, leading to requiring to go through the material again.

What is important to do while learning is to recognize when certain behaviors or circumstances make learning less efficient than desired and writing those behaviors or circumstances down so that you may have your own list of causes for learning less efficiently. As you discover those causes, make sure to put in place some contingency plans that will help you reduce their impact on your learning abilities.

Question

#

How often should I review my personal notes?

Answer

#

The simple answer is to do it as often as possible.

The more often you can go through your notes and review them, the more they will be fresh in your mind.

However, as you write more and more notes, it will be difficult to go through them all. At one point you will need to start prioritizing what you want to review and at what frequency. I see writing the same way I see programming: certain parts of the code work well and don’t need to be touched for a while, so they are left alone. However, sometimes certain parts have become neglected with time and need to be looked at again.

All my notes on this blog, as well as my private project and personal notes are committed to git automatically every time I switch out of VS Code. This means that I have a very granular history of the last time a file was touched. This allows me to look back at the list of files I have and to find the files that haven’t been touched in a while. Sometimes I’ll read them again, think about them a bit, then close them if I have nothing to add. Other times I’ll write about something that I’ve learned since then. Doing this feels a lot like improving some part of a codebase after you’ve learned how a piece of code would be used.

Another way you can schedule reviewing your notes would be with a spaced repetition system like Anki. It wouldn’t be too much work to have a system that automatically creates an Anki note/card with a link to the file itself. You would click on the link, which would open the file in your text editor of choice, read the note, change it if needed, then close it. You would then go back to Anki and pick between hard/good/easy to decide when is the next time you would review the note.

I’d suggest dedicating a small amount of your week going through what you wrote and refreshing your memory. It is definitely useful to simply write things to get them out of your mind and to attempt to express your thoughts clearly. It’s also useful to review what you thought about because your subconscious might have kept working on those thoughts well after you wrote them down and there might be more to contribute on a topic.