Mid Term Exam 2 (14th Batch) — Full Solution

Three questions, worth 4, 3, and 4 marks — gradient descent optimization, recovering a function from its derivative, and constrained optimization with Lagrange multipliers.

Question 1 — Gradient Descent Optimization

Suppose we apply gradient descent to minimize

$f(x,y) = x^2 + 4y^2 + 2xy - 4x + 6$

with learning rate $\alpha = 0.1$ and current parameter value $(x_0,y_0) = (2,-1)$.

1. Derive the general gradient descent update equation for $x$ and $y$.
2. Perform one iteration to compute $(x_1, y_1)$.
3. Compute $f(x_0,y_0)$ and $f(x_1,y_1)$.
4. State whether the update improved the objective.

Concept Needed

Gradient descent moves each parameter a small step opposite to the partial derivative of the objective at the current point — that direction is the one along which $f$ decreases fastest.

$x_{n+1} = x_n - \alpha \frac{\partial f}{\partial x}, \qquad y_{n+1} = y_n - \alpha \frac{\partial f}{\partial y}$

Given / Required

Given $f(x,y) = x^2+4y^2+2xy-4x+6$, $\alpha = 0.1$, $(x_0,y_0) = (2,-1)$. Required: the update rule, $(x_1,y_1)$, $f(x_0,y_0)$, $f(x_1,y_1)$, and whether the step improved the objective.

Step 1: Find the Partial Derivatives

$\frac{\partial f}{\partial x} = 2x + 2y - 4, \qquad \frac{\partial f}{\partial y} = 8y + 2x$

Step 2: Write the Update Equations

$x_{n+1} = x_n - \alpha(2x_n + 2y_n - 4)$

$y_{n+1} = y_n - \alpha(8y_n + 2x_n)$

Step 3: Evaluate the Gradient at $(2,-1)$

$\frac{\partial f}{\partial x}\Big|_{(2,-1)} = 2(2) + 2(-1) - 4 = 4 - 2 - 4 = -2$

$\frac{\partial f}{\partial y}\Big|_{(2,-1)} = 8(-1) + 2(2) = -8 + 4 = -4$

Step 4: Apply the Update Rule with $\alpha = 0.1$

$x_1 = 2 - 0.1(-2) = 2 + 0.2 = 2.2$

$y_1 = -1 - 0.1(-4) = -1 + 0.4 = -0.6$

So $(x_1,y_1) = (2.2,\ -0.6)$.

Step 5: Compute $f(x_0,y_0)$ and $f(x_1,y_1)$

At $(x_0,y_0) = (2,-1)$:

$f(2,-1) = (2)^2 + 4(-1)^2 + 2(2)(-1) - 4(2) + 6 = 4 + 4 - 4 - 8 + 6 = 2$

At $(x_1,y_1) = (2.2,-0.6)$:

$f(2.2,-0.6) = (2.2)^2 + 4(-0.6)^2 + 2(2.2)(-0.6) - 4(2.2) + 6$

$= 4.84 + 1.44 - 2.64 - 8.8 + 6 = 0.84$

Step 6: Did the Update Improve the Objective?

Since $f(x_1,y_1) = 0.84 < f(x_0,y_0) = 2$, the function value decreased.

Verification

Since the gradient direction $(-2,-4)$ points where $f$ decreases, and we moved opposite to it, a lower $f$ value is expected. This matches the computed answer, so the calculation is verified.

Question 2 — Finding a Function from its Derivative

If

$\lim_{h\to 0}\frac{f(x+h)-f(x)}{h} = 3x^2 - 4x + 1 \quad \text{and} \quad f(1) = 5,$

find $f(x)$. Hence determine the coordinates where the tangent is horizontal.

Concept Needed

The given limit is exactly the definition of $f'(x)$, so $f'(x) = 3x^2-4x+1$. Recovering $f(x)$ from $f'(x)$ means integrating, and the constant of integration is pinned down using the given point $f(1)=5$. A horizontal tangent occurs wherever the slope $f'(x)$ is zero.

Given / Required

Given $f'(x) = 3x^2 - 4x + 1$ and $f(1) = 5$. Required: $f(x)$, and the point(s) where the tangent line is horizontal.

Step 1: Integrate $f'(x)$ to get $f(x)$

$f(x) = \int (3x^2 - 4x + 1)\,dx = x^3 - 2x^2 + x + C$

Step 2: Use $f(1) = 5$ to Find $C$

$f(1) = (1)^3 - 2(1)^2 + (1) + C = 1 - 2 + 1 + C = 0 + C$

$5 = C \quad \Rightarrow \quad C = 5$

$f(x) = x^3 - 2x^2 + x + 5$

Step 3: Find Where the Tangent is Horizontal

A horizontal tangent means slope $=0$, i.e. $f'(x) = 0$:

$3x^2 - 4x + 1 = 0$

Factoring:

$3x^2 - 3x - x + 1 = 0$

$3x(x-1) - 1(x-1) = 0$

$(3x-1)(x-1) = 0$

$x = \frac{1}{3} \quad \text{or} \quad x = 1$

Step 4: Find the Corresponding $y$-coordinates

At $x = 1$:

$f(1) = 1 - 2 + 1 + 5 = 5$

At $x = \dfrac{1}{3}$:

$f\left(\frac13\right) = \left(\frac13\right)^3 - 2\left(\frac13\right)^2 + \frac13 + 5 = \frac{1}{27} - \frac{2}{9} + \frac{1}{3} + 5$

Converting to a common denominator of 27:

$= \frac{1}{27} - \frac{6}{27} + \frac{9}{27} + \frac{135}{27} = \frac{139}{27}$

Verification

Differentiating $f(x) = x^3-2x^2+x+5$ gives back $3x^2-4x+1$, which matches the given derivative — so the recovered function is correct.

Question 3 — Constrained Optimization using Lagrange Multipliers

Minimize

$f(x,y) = 2x^2 + 3y^2 + xy \quad \text{subject to} \quad x + 3y = 9$

1. Form the Lagrangian.
2. Compute the three first-order conditions.
3. Solve for $(x^*, y^*)$.
4. Find the minimum value of $f(x,y)$.

Concept Needed

Lagrange multipliers convert a constrained optimization problem into an unconstrained one by introducing a multiplier $\lambda$ for the constraint. At the optimum, $\nabla f = \lambda \nabla g$ — the gradients of the objective and constraint are parallel.

$\mathcal{L}(x,y,\lambda) = f(x,y) - \lambda\big(g(x,y) - k\big)$

Given / Required

Given $f(x,y) = 2x^2+3y^2+xy$, constraint $g(x,y): x+3y=9$. Required: $(x^*,y^*)$ and the minimum value of $f$.

Step 1: Form the Lagrangian

$\mathcal{L}(x,y,\lambda) = 2x^2 + 3y^2 + xy - \lambda(x + 3y - 9)$

Step 2: Write the First-Order Conditions

Setting all three partial derivatives to zero:

$\frac{\partial \mathcal{L}}{\partial x} = 0: \quad 4x + y - \lambda = 0 \quad \cdots (1)$

$\frac{\partial \mathcal{L}}{\partial y} = 0: \quad 6y + x - 3\lambda = 0 \quad \cdots (2)$

$\frac{\partial \mathcal{L}}{\partial \lambda} = 0: \quad x + 3y - 9 = 0 \quad \cdots (3)$

Step 3: Eliminate $\lambda$

From (1): $\lambda = 4x + y$. Substitute into (2):

$6y + x - 3(4x+y) = 0$

$6y + x - 12x - 3y = 0$

$3y - 11x = 0 \quad \Rightarrow \quad y = \frac{11x}{3} \quad \cdots (4)$

Step 4: Substitute into the Constraint

$x + 3\left(\frac{11x}{3}\right) = 9$

$x + 11x = 9$

$12x = 9 \quad \Rightarrow \quad x^* = \frac{3}{4}$

Step 5: Find $y^*$

$y^* = \frac{11}{3}\left(\frac{3}{4}\right) = \frac{11}{4}$

So $(x^*, y^*) = \left(\dfrac{3}{4},\ \dfrac{11}{4}\right)$.

Verification of the Constraint

$x^*+3y^* = \frac34 + \frac{33}{4} = \frac{36}{4} = 9 \quad \checkmark$

Step 6: Compute the Minimum Value of $f(x,y)$

$f\left(\frac34,\frac{11}{4}\right) = 2\left(\frac34\right)^2 + 3\left(\frac{11}{4}\right)^2 + \left(\frac34\right)\left(\frac{11}{4}\right)$

$2\left(\frac{9}{16}\right) = \frac{18}{16}, \qquad 3\left(\frac{121}{16}\right) = \frac{363}{16}, \qquad \frac{3}{4}\cdot\frac{11}{4} = \frac{33}{16}$

$f = \frac{18}{16} + \frac{363}{16} + \frac{33}{16} = \frac{414}{16} = \frac{207}{8}$

Since $f(x,y)$ is a positive-definite quadratic form (coefficients $2, 3$ positive and no dominating negative cross-term), the constrained stationary point found here is indeed a minimum.

Interview Corner