Interview with Sonar Python Developers Part 2

TL;DR overview

• The second part of Sonar's Python team interview explores security-focused analysis, including how taint analysis tracks untrusted data through Python applications to find injection vulnerabilities.
• The team explains the trade-offs between precision and recall in security rules, and how they reduce false positives for Python's dynamic patterns without missing real vulnerabilities.
• New security rules for Python are validated against both the OWASP benchmark and real open-source projects, ensuring they detect actual vulnerabilities rather than just synthetic test cases.
• Developers maintaining Python applications will benefit from understanding how SonarQube's security analysis works to better interpret and act on security findings in their own projects.

Welcome back to our two-part series where we pose questions from the Community to three Python developers from Sonar. 

As part of our celebration of all things Python during April we took the opportunity to sit down with Nafiul, Cheng, and Yaniv, three Python fans with different roles at Sonar, and get their take on questions submitted by our Community. You can check out Part 1 here.

What are your thoughts on Python tooling (IDEs, extensions, testing tools) and its maturity?

Quazi Nafiul Islam

Tooling is quite good in Python although it isn't as good as Java in some cases. I mainly use PyCharm as my main IDE, and I have a few plugins installed like AceJump and SonarQube for IDE; I don't use much else. I think the place where Python could see a lot of improvement is packaging and package management.

Cheng Chen

I think the tooling of Python is very, very mature. On the IDE side you have two big options: VS code and PyCharm… VS Code is very good and meets most needs. I was a VS code user, and now I'm using PyCharm and I love it! When I do a lot of scientific computing I also use Spyder which is a specialized IDE for that.

With a huge and dynamic community, Python has a ton of packages that you can explore for almost everything that you could think of, and why not wrap your codes into a package and share it in the community, so everyone can use it?

Yaniv Nizry

The Python community stands out as the largest among other programming languages, if I may say so, making it a highly mature programming language. It is evident when encountering a problem or aiming to write a specific code; chances are high that someone has already tackled it within the community. Due to the popularity of Python, there is a demand for great tooling and many companies are after that demand.

My personal environment is very dependent on my project, usually, I'll use a simple script so it will be one file and probably IDLE/VS code. In case I program a bigger project I'll move to a more organized manner using GIT, PyCharm, Pyvenv, etc.

What's a recent change to Python (the programming language itself) that gets you excited?

Quazi Nafiul Islam

It’s not just one thing, but the fact that Python is constantly getting better. For example, we now have PyScript, which allows you to build front-end applications with Python. Pandas has a sort-of upgrade with a new library called Polars which allows you to do everything that you can do in Pandas but much faster. However, most importantly, the CPython interpreter itself is getting faster and faster with each release.

Here is a Tweet from PyPI regarding how much compute they were saving when they upgraded to Python 3.11, which introduced many performance enhancing features.

Cheng Chen

My first thought was the introduction of dataclasses in Python 3.7, but then I think the type hints from Python 3.5 are a more impactful change. Type hints greatly improve the code readability and let you spot mistakes easier before running the code. Several popular static type packages like mypy and pydantic also directly benefit from it. This complements Python's flexibility as a dynamically typed language.

Python type hints have been constantly evolving since their introduction and something that I liked recently is the explicit type alias and the simplified type union annotation introduced in 3.10. I'll give an example in the ML world. In ML we talk a lot about vectors, labels, samples, datasets…

A vector is a list of float or int numbers:

Vector: TypeAlias = list[float | int]

A label could be a boolean, an integer, or a string. A label could also be missing (None):

Label: TypeAlias = bool | int | str | None

A data sample is a tuple of a vector and a label:

DataSample: TypeAlias = tuple[Vector, Label]

A dataset is a list of data samples:

Dataset: TypeAlias = list[DataSample]

Then we define a method train_model which takes in a dataset as argument and train the model, I can write

def train_model(ds: Dataset) -> Model:
  pass

Without explicit type alias, I would write (a bit harder to read):

def train_model(
ds: List[Tuple[List[Union[float,int]], Union[bool,int,str,None]]]
) -> Model:
  pass

I know we are working to leverage Python type hints into our Sonar solution. This will be great, as I believe that knowing (somehow) the types of variables and methods will help the analysis be even more accurate and efficient.

Yaniv Nizry

I don't follow updates and changes in Python. For me, Python already has everything I need 😉.

Wrap up

A big thank you to Nafiul, Cheng, and Yaniv, three of Sonar's very own Python developers. 

We discovered their views on tooling and what recent changes have excited them the most (and that Yaniv already has everything he needs 😀).

At Sonar we want to deliver solutions that add value to developers and help them create Clean Python Code. Follow us to learn more, or download our free and open-source plugin SonarQube for IDE from your favorite IDE marketplace to try it for yourself.

Bios

Nafiul Islam, programming since 14, has a decade of software experience. Adept in Python and exploring Rust, he authored "Mastering PyCharm" at 21. Nafiul has spoken at global Python conferences and held positions at JetBrains and Microsoft. In his free time, he loves reading fantasy novels. Follow him @gamesbrainiac on Twitter.

Cheng Chen trained as a computer scientist, Cheng has been working in machine learning and artificial intelligence since 15 years ago in different industry branches such as computer vision, FMCG, digital manufacturing, and biometrics. He recently joined Sonar to explore ML for code analysis.

Yaniv Nizry is a Vulnerability Researcher at Sonar where he leverages his expertise to identify and mitigate vulnerabilities in complex systems. Starting his way as a software engineer, he shifted his focus while serving in the IDF's 8200 unit, where he gained experience in both offensive and defensive cybersecurity tactics.

Enjoy all things Python, and want more? Register now for our upcoming webinar Code Quality for your Python projects, with Nafiul Islam - Wednesday, May 10th - 5 PM CEST / 10 AM CDT.