Notion.so I use Notion.so to track my day to day tasks and their progress. I also use it to write down any idea I have during the day that I would like to explore at some point in the future. I’ve described my task management system in more details.
JIRA I work in a corporate environment, which means that we need to track tasks and assignments through a shared system. That system is JIRA. I don’t particularly like JIRA after having used redmine for 5+ years and liking it. JIRA is slow, complex and cumbersome, which makes me avoid it at all costs. It’s a shame, given that task management is one of the most important things in a software development business.
PyCharm My current job is mostly about writing python code, and PyCharm is the best IDE to do that. I use it to implement new functionality, write tests and debug issues. It is highly customizable which makes using it a joy. I also make use of the run/debug configurations regularly to test a variety of cases over the lifetime of the projects I work on. Finally, because I’ve used PHPStorm in the past, transitioning from PHPStorm to PyCharm was easy and painless.
iTerm2 I spend a good part of my day interacting with various CLIs. I use a Mac and the terminal simply doesn’t cut it for me and I prefer interacting with GUI tabs over having a tmux.
Chrome Anyone doing software development spends a large portion of their time online looking for solutions to their problems/questions. I use Chrome mostly to use the Google suite, to use Jira/Confluence or to look at reports generated by the tool I work on.
Google suite (Gmail, Docs, Sheets, Slides) I work in a collaborative environment which means writing and reading documents to share ideas, data, requirements and notes with others.
Slack Working with others means communicating with others. While it’s possible to talk directly to my teammates, sometimes it is more respectful to simply send them an instant message asking them to come to see you when they are available. Slack also allows communication with people in other teams, as well as people that are in separate offices.
GitHub We use git exclusively at work as a way to do version control. We use GitHub as our central location to share our repository, as well as to do code reviews.
Drone Whenever I push code to GitHub, Drone makes sure that a variety of tasks are executed to ensure code quality: linting, code formatting, type checking, unit/function/integration tests and documentation building. Having a build system reduces the time between faulty code is pushed to GitHub and the code being fixed.
If you have masks already, wear one. It will protect you from others and others from you.
The official answer from the Government of Canada and the CDC is that wearing a face mask is not an effective solution to protect yourself against the virus.
If you are sick and you need to go out in public, wearing a face mask will help reduce the chances of spreading the virus to others.
If you want to avoid being ill due to the virus, the best way to do so is to avoid being exposed to the virus. This means staying at home in isolation.
When using skills in which you can make mistakes, it is important to monitor what you do and where you make your mistakes. Like any performance optimization problem, you want to figure out where you make the most mistakes and where you’ll benefit the most from fixing those mistakes. If you make the same mistakes 100 times at the cost of a minute each time, that may be preferable to making 1 mistake that cost 100 minutes to fix, given that once the mistake happens, the cost stays the same.
Document your skills. Write down what you do, when you do it (triggers) and what kind errors you make during those steps. Evaluate how many times you make that mistake and how long it costs to fix. Then when you use your skills, track when you make mistakes and how long it takes you to fix the mistake.
As an example, think of a software engineer doing code reviews. Reviewing code requires going through a variety of checks: is the build passing? is the functionality properly implemented? are there tests? are the new files in the proper location? Without a list, the engineer is left looking at the code without any clear checklist. If he is methodical he will have a list he goes through in his head. If not, then he will most likely only look at the code and give it a summary opinion, that is, whether he likes what he sees, or not.
Given a non-explicit methodology it is hard to assess where the mistakes are made and which mistakes cost the most to fix. If you don’t check that tests were written for the new functionality or changes, what impact will it have in the future? Depending on how likely the code is to change, the likelihood that something gets broken may be significant. While adding a test may require a few minutes, one has to judge how much time would be wasted if a bug were introduced in the piece of code. As code complexity increases, so does the cost of fixing issues in that code.
With an explicit checklist you will be able to track the things that you want to verify before code is merged. As your checklist covers more and more cases, this list will reduce the likelihood that the person who wrote code made a mistake that gets to production. By the same token it will increase your effectiveness as a software engineer to produce quality code.
If someone has written a blog for example, it is possible to figure out a variety of information about them simply based on what is written in those articles.
If someone writes about a specific software, we may not be able to infer right away what they think about said software, but we know that they’ve spent enough time to learn a bit about this specific software. If we know all the other alternatives in this category of software, we could infer that the user believes that this software is possibly better than the alternatives, otherwise why would they have picked it?
If the writer writes a lot on a topic, that is also a clue about their beliefs. They probably think that this topic is important, hence why they write about it. Maybe they write about this topic because it is lucrative to them.
What the writer doesn’t write about is also informative. If they write mostly about technology, maybe they don’t care about politics or sports?
It is possible to extract if-then rules from their writings, which generally expresses some form of believe that if something then something else. There are other variants where only if is provided (if something, something else, or something else, if something).
A writer may use certain adjectives to describe things as “easy”, “simple”, “straightforward”, “difficult”, “impossible”, “hard”, etc. Those are also useful of indicators of the writer’s beliefs.
Nowadays with LLMs it’s easy to provide a prompt such as “What beliefs are expressed in the following text” or “Extract the beliefs in the following text”. With a bit of scripting we can extract everything available on the blog and feed it to a LLM to let it extract the beliefs for us.
Use Pandas Profiling to quickly generate a document that will provide you with a first overview of the data.
Your first step should be to look for warnings and messages at the top of the document. Look for entries about missing values, those will point you to variables that may need attention during the data cleaning and data imputation phases of your machine learning problem. As you are doing an assessment, simply indicate that data is missing in these variables and then see if you can determine why by looking at a few examples by loading the data in a pandas dataframe.
Are there a lot of duplicated rows? Depending on the data you’ve been provided, this may help you identify whether or not something is wrong with the data you were provided. If all entries are supposed to be unique because they represent a single (entity, timestamp, target) tuple, then you should ask yourself why it isn’t the case. Is it possible that the dataset was created by appending a collection of other documents, leading to duplicate lines? If so, you may have to do some dataset preprocessing in order to get rid of duplicate rows.
Look for variables that are indicated as highly correlated with other variables. High correlation means that it may be possible that one variable has exactly (or almost) the same values as the other variable, which would provide little information to a machine learning model. It would also mean that picking one variable out of two correlated variables would avoid the cost of storing both.
Look at each variable in turn and view its details.
Look at the distribution of values. Are they uniformly distributed, normally distributed, binomially distributed, etc.?
If there are only two possible values for a variable, are those values approximately the same or one value is dominant compared to the other? Were you to try and predict this variable, you would have to deal with class imbalance.
Are the values of the variables sensible to you? Are variables composed of multiple information, such as the value and the unit used for the measurement? You would generally prefer composite values to be separated into different variables as it will be easier to process using machine learning models.
When looking at numbers distribution, are there outliers (values that are either a lot smaller or larger than the rest)? It is sometimes important to ask those who provided you with the data if they can explain those outliers. In general you will want to ignore outliers during training as they may skew your model toward them, resulting in less than ideal results for all the other data points.
The quality of a dataset is inversely proportional to the number of operations you need to apply to it to make it a clean dataset. That is to say that if you don’t need to do anything on the data provided to you, then it is a good dataset.
The following list is ordered according to the importance I give to those skills in my success at work.
Analysis/understanding the problems: There’s no point in doing work if you don’t know why you’re doing it. Understanding why you are solving a specific problem will give you insights on the best way to solve the problem itself using the right approach.
Programming: I’m a software developer, so I spend most of my time (or hope I am) on writing code. That also includes reading code written by others.
Prioritization: I need to decide what tasks are more important than other tasks and organize the list of tasks from most to least important. At my current job, it’s everyone’s responsibility to decide what is important to build.
Time management: I have a variety of responsibilities so I need to juggle between them and the time I allocate to them. I also need to accomplish certain tasks according to deadlines, so it is important to properly manage my time, what gets done and when it gets done.
Design/code organization: Programming is more than science. You also need to think about how the different parts of the code interact with each other and how to organize the code so that it is easy for others to understand the code and to participate in its development.
Negotiation: Working in an organization means negotiating with others to make your ideas heard, accepted and developed. Not everyone will agree with your ideas, so you need to spend time and effort to convince others that your idea is the best.
Scheduling: While time management is about making sure that you’re spending your time on a task at the right moment, scheduling is about organizing your use of time with others in the company. It is about scheduling meetings and or events.
Delegation: Being able to offload work to others is important. It is however a difficult skill because you need to be able to properly communicate with others what needs to be done and the expectations that you have (or that others have) regarding the completion of the task.
Debugging: I love playing detective when there are issues in the code I maintain. I’ve acquired over the years a good ability to understand a system and to quickly pinpoint the cause of a bug or low performing system.
Communication: Work is largely exchanging information and aligning with your coworkers. The most important skill of communication is listening because you want to understand first before saying anything.
When joining a new project without tests, here is the value you need to provide through the addition of tests:
the application works and doesn’t crash
the application works and supports a few input cases
the application works and supports a variety of input cases
the application works and is robust to most input cases
Start by finding the main entrypoint of the program and call it in a test. Your test doesn’t have to do much, other than ensuring you can start the program and possibly exit. Your goal should not be to assert anything yet, but to exert the code. Create a few tests that do very few things other than starting and terminating the program. Once you’ve covered a few use cases, you can use those tests to ensure that the application can start, do a few things, then terminate without crashing.
Start unit testing the various parts of the code that are critical. To determine what is critical, you’ll have to dig into the code. With the tests you initially wrote you will get a sense of the “critical” pieces, simply due to the fact that they are executed whenever the program starts and stops, which are two things you always want to work.
When writing unit tests, always aim to write a test case that covers the happy path first. You want to demonstrate that the functionality a class or function is supposed to provide is there first and foremost. Then you want to test its robustness and its ability to handle a variety of input cases. Given a large codebase, start by covering most of the code with the happy paths before you start to dig into the special cases.
Start by asking your teammates to assess whether they are at their peak productivity or not. If they already are, ask them what you could do to help them be more productive. If they are not at their peak productivity, ask them what is making them unproductive. Make it your goal to help your teammates be as productive and efficient as possible.
Being unproductive is like slow software. If you want to improve it, you have first to start with profiling the program to have an idea of what is making it slow. Once you have a general idea of the cause of the slowness, you will want to focus on your biggest sources of slowness because they are likely to be the ones you can help improve.
In software development, the biggest source of productivity loss is meetings. Meetings break the flow of developers, they consume time and are generally easy to replace with a properly written document. If you can act as the proxy for multiple members of your team so that they do not have to attend all the same meetings you are attending, you can help them reduce this meeting fatigue.
Another source of ineffectiveness is back and forth code reviews. If you need to do multiple iterations of review to close a PR, this PR was too big. If you also end up having to do more than two back and forth, you should simply reach out to the author of the PR and do a pair programming session with them to resolve most of the issues you have with the PR in a single session.
Code depends on other code. It should be your first priority to make sure that code that is developed by others is not stuck in the PR queue for a long period of time because the author might be building on that code. The code in that PR may actually be used by code the author is already writing while waiting for the PR to be reviewed. If he needs to go back and fix things because of delays in the PR, this will make his whole development process inefficient. Having to wait or being blocked is an important source of productivity loss.
Help others avoid working on too many tasks at once. The brain does not handle multitasking very well and it generally leads to lower performance and higher error rate.
Notion.so I use Notion.so to track my day to day tasks and their progress. I also use it to write down any idea I have during the day that I would like to explore at some point in the future. I’ve described my task management system in more details.
JIRA I work in a corporate environment, which means that we need to track tasks and assignments through a shared system. That system is JIRA. I don’t particularly like JIRA after having used
redmine for 5+ years and liking it. JIRA is slow, complex and cumbersome, which makes me avoid it at all costs. It’s a shame, given that task management is one of the most important things in a software development business.
Visual Studio Code I use VS Code to take notes throughout the day as well as to edit files from time to time.
PyCharm My current job is mostly about writing python code, and PyCharm is the best IDE to do that. I use it to implement new functionality, write tests and debug issues. It is highly customizable which makes using it a joy. I also make use of the run/debug configurations regularly to test a variety of cases over the lifetime of the projects I work on. Finally, because I’ve used PHPStorm in the past, transitioning from PHPStorm to PyCharm was easy and painless.
iTerm2 I spend a good part of my day interacting with various CLIs. I use a Mac and the terminal simply doesn’t cut it for me and I prefer interacting with GUI tabs over having a tmux.
Chrome Anyone doing software development spends a large portion of their time online looking for solutions to their problems/questions. I use Chrome mostly to use the Google suite, to use Jira/Confluence or to look at reports generated by the tool I work on.
Google suite (Gmail, Docs, Sheets, Slides) I work in a collaborative environment which means writing and reading documents to share ideas, data, requirements and notes with others.
Slack Working with others means communicating with others. While it’s possible to talk directly to my teammates, sometimes it is more respectful to simply send them an instant message asking them to come to see you when they are available. Slack also allows communication with people in other teams, as well as people that are in separate offices.
GitHub We use git exclusively at work as a way to do version control. We use GitHub as our central location to share our repository, as well as to do code reviews.
