03 Function properties#
The expression \(\displaystyle \lim _{x\to \pm \infty } f(x)\) represents the limit of the function \(f(x)\) as \(x\) approaches positive or negative infinity.
\(\displaystyle \lim _{x\to +\infty } f(x)\): This represents the limit of the function \(f(x)\) as \(x\) approaches positive infinity. It describes the behavior of \(f(x)\) as \(x\) gets larger and larger.
\(\displaystyle \lim _{x\to -\infty } f(x)\): This represents the limit of the function \(f(x)\) as \(x\) approaches negative infinity. It describes the behavior of \(f(x)\) as \(x\) gets smaller and smaller (more negative).
These limits are used to understand the behavior of functions at extreme values of \(x\). If the limits exist and are finite, it gives us information about the end behavior of the function. If the limits are infinite or do not exist, it tells us that the function does not settle down to any particular value as \(x\) gets extremely large or extremely small.
Note In mathematics, \(e\) is a special number that is approximately equal to 2.71828. It is an irrational number, meaning it cannot be expressed as a simple fraction, and its decimal representation never ends or repeats.
Question 1 (ReLU)#
The function \(f(x) = \max(0, x)\) is also known as the ReLU (Rectified Linear Unit) function in the field of machine learning. It returns the input value if it’s positive, otherwise it returns 0.
When we consider the limit of \(f(x)\) as \(x\) approaches negative infinity, we are looking at the behavior of the function as \(x\) gets very large in the negative direction. For all negative values of \(x\), \(f(x) = \max(0, x)\) will return 0, because 0 is greater than any negative number. Therefore, the limit of \(f(x)\) as \(x\) approaches negative infinity is 0.
So, \(\displaystyle \lim _{x\to {\color{yellow}{-}} \infty } f(x) = 0\).
When we consider the limit of \(f(x)\) as \(x\) approaches positive infinity, we are looking at the behavior of the function as \(x\) gets very large in the positive direction. For all positive values of \(x\), \(f(x) = \max(0, x)\) will return \(x\), because \(x\) is greater than 0. As \(x\) approaches positive infinity, \(f(x)\) also approaches positive infinity.
Therefore, the limit of \(f(x)\) as \(x\) approaches positive infinity is positive infinity.
So, \(\displaystyle \lim _{x\to {\color{yellow}{+}} \infty } f(x) = \infty\).
Derivative and Differentiable#
The function \(f(x) = \max(0, x)\) is differentiable for all real numbers except at \(x = 0\).
Here’s why:
For \(x > 0\), \(f(x) = x\), which is a linear function and is differentiable everywhere. The derivative is \(f'(x) = 1\).
For \(x < 0\), \(f(x) = 0\), which is a constant function and is also differentiable everywhere. The derivative is \(f'(x) = 0\).
However, at \(x = 0\), the function is not differentiable. This is because the derivative from the left (which is 0) does not equal the derivative from the right (which is 1).
So, the function \(f(x) = \max(0, x)\) is differentiable for all \(x \neq 0\).
Question 2 (Binary Classification)#
The function \(f(x) = \frac{1}{1+e^{-x}}\) is known as the logistic function, which is often used in machine learning and statistics for binary classification problems.
When we consider the limit of \(f(x)\) as \(x\) approaches negative infinity, we are looking at the behavior of the function as \(x\) gets very large in the negative direction. As \(x\) approaches negative infinity, \(e^{-x}\) approaches positive infinity. Therefore, the denominator \(1+e^{-x}\) also approaches positive infinity, and the whole fraction \(\frac{1}{1+e^{-x}}\) approaches 0.
So, if you have a fraction where the numerator is 1 and the denominator is infinity (an incredibly large number), the fraction is essentially zero.
So, \(\displaystyle \lim _{x\to {\color{yellow}{-}} \infty } f(x) = 0\).
When we consider the limit of \(f(x)\) as \(x\) approaches positive infinity, we are looking at the behavior of the function as \(x\) gets very large in the positive direction. As \(x\) approaches positive infinity, \(e^{-x}\) approaches 0. Therefore, the denominator \(1+e^{-x}\) approaches 1, and the whole fraction \(\frac{1}{1+e^{-x}}\) approaches 1.
So, \(\displaystyle \lim _{x\to {\color{yellow}{+}} \infty } f(x) = 1\).
These limits reflect the fact that the logistic function has horizontal asymptotes at \(y=0\) and \(y=1\).
Derivative and Differentiable#
The function \(f(x) = \frac{1}{1+e^{-x}}\) is differentiable for all real numbers.
Here’s why:
The function is composed of elementary functions (exponential, addition, division) which are all differentiable. The composition of differentiable functions is also differentiable.
Furthermore, the denominator of the fraction, \(1+e^{-x}\), is always greater than 0 for all real \(x\), so we don’t have any division by zero issues which could have caused problems with differentiability.
So, the function \(f(x) = \frac{1}{1+e^{-x}}\) is differentiable for all real numbers \(x\).