Inside of mathematical logic, there are philosophical positions, and people like me, who follow these things, get peculiarly emotionally attached to various positions. At the moment, a moderately high-status person-on-the-internet is claiming that second-order logic is NECESSARY for us to talk about the integers. Clearly, I am convinced that they are wrong-on-the-internet. In working out my (irrational, ridiculous) emotions, I am writing a blog post.
One of the standard things to do in mathematics is to take an existing theory, which generally includes some "things" - such as points and lines, or integers - and extend it with new concepts. For example, it's feasible to do geometry talking about points and lines and points being on lines without ever mentioning "between". However, you might want to talk about betweenness, and so you add the vocabulary and some definitional rules, and then work in this richer system.
You can create a nice sequence, an idealized history of numbers, where you start with the positive integers, then extend it with zero (a new number), then extend it with either negatives or fractions (new kinds of numbers), then the other one leading to both negatives and fractions, then some principle that gives you the square root of two then perhaps some principle that gives you some other interesting number like pi. At this point, you might consider adding infinity and/or infinitesimals, which might be appropriate for a calculus class based on Abraham Robinson's analysis. Or more likely, you go to the complex numbers by introducing a new kind of number, the so-called "imaginary" numbers.
Sometimes, mathematicians discover theorems after extending the system, and due to the complexity of the proof and how entangled it is with the extension, become convinced that the extension was necessary to the conclusion. Historically, this happened with the prime number theorem - the first proof was via complex analysis, and Hardy (a competent mathematician) believed that complex analysis was necessary for the proof. However, it turned out that it was entirely possible to prove the prime number theorem directly, without extending the integers to the complex plane. In analogous cases, it has generally turned out that it is possible to find an elementary (that is, without extensions) proof.
Why is it generally possible to find an elementary proof? One reason is that mathematicians, in crafting extensions, generally try to craft conservative extensions - meaning extensions that do not introduce new facts in the old concepts. A conservative extension means that it is never necessary to use the new concepts to prove a fact about the old concepts.
Harvey Friedman has written about 'demonstrably necessary uses of abstraction'. I admit I don't understand half of what he's doing, but he lays out a template for arguing that an extension is necessary to a particular conclusion. First, formalize your starting system. Second, demonstrate that the conclusion cannot be proven in the starting system (for example, by exhibiting a countermodel). Third, formalize your extension, and demonstrate that the conclusion can be proven in the new system (for example, by exhibiting a proof).
Without this sort of rigor, handwaving about a concept being necessary is dubious. The reasoning "I am an expert, and I can't think of how it could be done without X, therefore X is probably necessary." is an instance of Clarke's first law: "When a distinguished but elderly scientist states that something is possible, they are almost certainly right. When they state that something is impossible, they are very probably wrong."