Recently, the Spyder IDE faced serious typing delay or lag. Basically, after entering something using the keyboard, it takes almost half a second for it to appear on the screen. The issue can also be described as a typing lag in Spyder.
This seems to have been triggered by the installation of Big Sur OS.
Spyder slow on Mac
After intensive googling, we stumbled upon this Github page detailing several possible solutions.
The one that worked for us was installing pyqt and pyqtwebengine. Basically, type the following commands in the terminal:
pip install PyQt5
pip install PyQtWebEngine
The above solution should be very safe since it is just installing Python packages.
The above solution helped to solve the troubling issue of Spyder lagging. Since Spyder uses Qt for its GUI, it is critical to keep the various Qt related packages updated / at the correct version. This may be the reason why installing PyQt5 and PyQtWebEngine helps to remove the lag in Spyder.
Spyder very slow
There seems to be many reasons, other than the above, that can result in Spyder being very slow. One tip that is useful, is never update to the latest version of Spyder, Mac OS, or Anaconda immediately once it is released, unless it is absolutely necessary. Most of the bugs appear in the newest releases, and can cause multiple problems including making Spyder very slow. By updating at a later date, most of the bugs would have been solved by then and it is a much safer approach.
Previously, updating to the latest Spyder 4.1.5 also caused several problems, including lag, slowness or even Spyder simply just crashing.
Matplotlib is a popular plotting package used in Python. There are some things to note for plotting multiple figures, in separate windows.
A wrong approach may lead to matplotlib showing a black screen, or plotting two figures superimposed on each other, which may not be the desired outcome.
Sample Matplotlib Code for Plotting Multiple Figures in Separate Windows
import matplotlib.pyplot as plt
# plotting function here, e.g. plt.hist()
# plotting function here, e.g. plt.hist()
One way to do this is to use the plt.figure() command for each figure that you want to plot separately. Optionally, you can use plt.savefig() if you wish to save the figure plotted to the working directory folder.
At the end, use the plt.show() command. The plt.show() command should only be used once per script.
Spyder is a Python IDE that is bundled together with the Anaconda distribution.
There are some problems that are commonly faced when it comes to updating Spyder. One way to update Spyder is to open Anaconda Navigator and click the settings button which has an option to update Spyder. But the problem is that the process can take a very long time. The process shows that it is “loading packages of /User/…/opt/anaconda3”.
Updating Spyder is constricted by …
Another way to update Spyder is to type “conda update spyder” in the terminal. A problem that can crop up is the error message: “updating spyder is constricted by …”
Anaconda stuck updating Spyder [Solved]
For my case, it turns out that the version of Anaconda Navigator is outdated. Hence, I first updated Anaconda Navigator to the latest version.
Then, instead of clicking “Update application” which still didn’t quite work, we click on “Install specific version” and choose the latest version of Spyder (Spyder 4.1.5 in this case).
Then, the updating of Spyder in Anaconda Navigator worked perfectly!
Basically, if your code prints too much output to the console, it will slow down RStudio and your R code as well. It may be due to all the output clogging up the memory in RStudio. R is known to be “single thread” so it can only use 1 CPU at a time, even if your computer has multiple cores.
Hence, the tips are to:
Reduce the number of print statements in the code manually.
Set quiet=TRUE in all scan statements. Basically, the default behavior is that scan() will print a line, saying how many items have been read.
This is especially true with for loops, since the amount of printed output can easily number to the millions, and overwhelm RStudio.
When the Mac (or MacBook) is running for a long time, it is very liable to do one of the following things:
The problem is that your Python program or R program running in the background will most likely stop completely. Sure, it can resume when you activate the Mac again, but that is not what most people want! For one, it may impact the accurate calculation of elapsed time of your Python code.
Changing settings via System Preferences -> Energy Saver is a possible solution, but it is troublesome and problematic:
Have to switch it on and off again when not in use (many steps).
Preventing sleep may still run into screen saver, screen lock, etc.
Vice versa, preventing screen lock may still run into Mac sleeping, etc.
The solution is to install this free App called Amphetamine. Despite its “drug” name, it is a totally legitimate program that has high reviews everywhere. What this app does is to prevent your Mac from stopping, locking or sleeping. Hence, whatever program you are running will not halt till the program is done (or when you switch off Amphetamine).
It is a great program that does its job well! Highly recommended for anyone doing programming, video editing or downloading large files on Mac.
MacOS Catalina was released on October 7, 2019, and has been causing quite a stir for Anaconda users. Apple has decided that Anaconda’s default install location in the root folder is not allowed. It moves that folder into a folder on your desktop called “Relocated Items,” in the Security folder. If you’ve used the .pkg installer for Anaconda, this probably broke your Anaconda installation. Many users discuss the breakage at https://github.com/ContinuumIO/anaconda-issues/issues/10998.
Most of the time, users of R and Python will rely on packages and libraries as far as possible, in order to avoid “reinventing the wheel”. Packages that are established are also often superior and preferred, due to lower chance of errors and bugs.
We list down the most popular and useful packages in R and Python for data science, statistics, and machine learning.
This article is suitable for solving the following few problems:
module ‘sklearn.tree’ has no attribute ‘plot_tree’
pip install (on Spyder, Anaconda Prompt, etc.) does not install the latest package.
The leading reason for “module ‘sklearn.tree’ has no attribute ‘plot_tree” is because the sklearn package is outdated.
Sometimes “pip install scikit-learn” simply does not update the sklearn package to the latest version. Type “print(sklearn.__version__)” to get the version of sklearn on your machine, it should be at least 0.21.
The solution is to force pip to install the latest package:
pip install --no-cache-dir --upgrade <package>
In this case, we would replace <package> by “scikit-learn”.
Sometimes, pip install does not work in the Spyder IPython console, it displays an error to the effect that you should install “outside the IPython console”. This is not normal (i.e. it should not happen), but as a quick fix you can try “pip install” in Anaconda Prompt instead. It is likely that something wrong went on during the installation of Anaconda, Python, and the long-term solution is to reinstall Anaconda.
In the R language, often you have to convert variables to “factor” or “categorical”. There is a known issue in the ‘caret’ library that may cause errors when you do that in a certain way.
The correct way to convert variables to ‘factor’ is:
trainset$Churn = as.factor(trainset$Churn)
In particular, “the train() function in caret does not handle factor variables well” when you convert to factors using other methods.
Basically, if you use other ways to convert to ‘factor’, the code may still run, but there may be some ‘weird’ issues that leads to inaccurate predictions (for instance if you are doing logistic regression, decision trees, etc.)
To elaborate, clf is your Decision Tree classifier (to be defined before plotting the tree):
# Example from https://scikit-learn.org/stable/modules/generated/sklearn.tree.plot_tree.html
clf = clf.fit(iris.data, iris.target)
The outcome is a Vector Graphics format (.eps) tree that will retain its full resolution when zoomed in. The bbox_inches=”tight” command prevents truncating of the image. Without that command, sometimes the sklearn tree will just be cropped off and be incomplete.
R has the package “psych” which allows one to calculate the Cronbach’s alpha very easily just by one line:
For Python, the situation is more tricky since there does not seem to exist any package for calculating Cronbach’s alpha. Fortunately, the formula is not very complicated and it can be calculated in a few lines.
An existing code can be found on StackOverflow, but it has some small “bugs”. The corrected version is:
The above code will run two Python files “testing.py” and “testingb.py” simultaneously. Add more python scripts if needed. The first line is called the “shebang” and signifies the computer to run bash (there are various versions but according to StackOverflow the above one is the best).
The above bash file can be saved to any name and any extension, say “bashfile.txt”.
Step 2 is to login to Terminal (Mac) or Putty (Windows).
chmod +x bashfile.txt
This will make the “bashfile.txt” executable.
Follow up by typing:
This will run the “bashfile.txt” and its contents. The output will be put into a file called “nohup.out”. The “nohup” option is preferred for very long scripts since it will keep running even if the Terminal closes (due to broken connection or computer problems).
If your child is interested in a Computer Science/Data Science career in the future, do consider learning Python beforehand. Computer Science is getting very popular in Singapore again. To see how popular it is, just check out the latest cut-off point for NUS computer science, it is close to perfect score (AAA/B) for A-levels.
According to many sources, the Singapore job market (including government sector) is very interested in skills like Machine Learning/ Deep Learning/Data Science. It seems that Machine Learning can be used to do almost anything and everything, from playing chess to data analytics. Majors such as accountancy and even law are in danger of being replaced by Machine Learning. Python is the key language for such applications.
The course includes many things from the basic syntax to advanced visualization of data. It teaches at quite a basic level, I am sure most JC students (or even talented secondary students) with some very basic programming background can understand it.
The best programming language for data science is currently Python. Try not to learn “old” languages like C++ as it can become obsolete soon. Anyway the focus is on the programming structure, it is more or less universal across different languages.
Students trying to import the package “matplotlib” on PyCharm will soon face the cryptic error message: “Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework.”
It is extremely puzzling what to do. I have researched the steps that you can follow to solve it in 3 minutes:
Recently, I am thinking of learning the Python language for Math programming.
An advantage for using Python for Math Programming (e.g. testing out some hypothesis about numbers), is that the Python programming language theoretically has no largest integer value that it can handle. It can handle integers as large as your computer memory can handle. (Read more at: http://userpages.umbc.edu/~rcampbel/Computers/Python/numbthy.html)
Other programming languages, for example Java, may have a maximum integer value beyond which the program starts to fail. Java integers can only have a maximum value of , which is pretty limited if you are doing programming with large numbers (for example over a trillion). For instance, the seventh Fermat number is already 18446744073709551617. I was using Java personally until recently I needed to program larger integers to test out some hypothesis.
How to install Python (free):
Hope this is a good introduction for anyone interested in programming!
Get a comprehensive, in-depth introduction to the core Python language with this hands-on book. Based on author Mark Lutz’s popular training course, this updated fifth edition will help you quickly write efficient, high-quality code with Python. It’s an ideal way to begin, whether you’re new to programming or a professional developer versed in other languages.
Complete with quizzes, exercises, and helpful illustrations, this easy-to-follow, self-paced tutorial gets you started with both Python 2.7 and 3.3— the latest releases in the 3.X and 2.X lines—plus all other releases in common use today. You’ll also learn some advanced language features that recently have become more common in Python code.
Explore Python’s major built-in object types such as numbers, lists, and dictionaries
Create and process objects with Python statements, and learn Python’s general syntax model
Use functions to avoid code redundancy and package code for reuse
Organize statements, functions, and other tools into larger components with modules
Dive into classes: Python’s object-oriented programming tool for structuring code
Write large programs with Python’s exception-handling model and development tools
Learn advanced Python tools, including decorators, descriptors, metaclasses, and Unicode processing