• bushvin@lemmy.world
    link
    fedilink
    arrow-up
    88
    arrow-down
    2
    ·
    9 months ago

    There is nothing more important than security patches on a system.

    I used to work at an FMI, which’s motto was “keep things stable”. Even the ciso department bought that crap. Until we hired a white hat hacker. The only thing given was the name of the company. He managed to get into the building, access an employee’s workstation and install a root kit on one of the most important financial message tracking systems (you know, the one that instructs other systems to transfer money), using a security bug, which would have been patched if they kept a regular (security) update cycle. After shit hit the fan, many people were fired and an update cycle was introduced.

    No system is important enough to not patch. And if you believe it is, you’re wrong.

    • 0x4E4F@lemmy.dbzer0.comOP
      link
      fedilink
      arrow-up
      21
      arrow-down
      12
      ·
      edit-2
      9 months ago

      Yeah, but that just takes way too much work. You think I really care about the company’s/bank’s money if I’m not getting paid enough for that job? Security patches can also introduce new problems, like x changes, so y doesn’t work, so the main app doesn’t work… and what, then I have to manually edit code, introduce the thing that x relied on so that y can work again?

      I’m sorry, but this is not your average IT department’s job… or if it is, I expect a damn good compensation for it.

      I’ve updated and rolled back snapshots because of shit like this… nah, not gonna try and figure out what the problem was… at least not for the salary I’m currently getting paid. If it burns, it burns, so be it.

      • li10@feddit.uk
        link
        fedilink
        English
        arrow-up
        20
        ·
        9 months ago

        I’d be surprised if you actually saw anything change from security updates tbh, I don’t think I’ve ever seen anything break from a quick patch.

        Dist upgrades are when things might break, but they’re only once every few years. Leave them too long though and you may end up with compatibility issues if you need to make changes.

        Fair enough if you’re not getting paid enough, the company should hire more people to stay on top of that though.

        • 0x4E4F@lemmy.dbzer0.comOP
          link
          fedilink
          arrow-up
          1
          ·
          9 months ago

          They have and that is why I don’t do them any more. Happened a few months ago in fact. Updated one of the Debian servers for one of webapps we have running (a black box piece of shit VM that’s stuck in 2010 I think)… suddenly, the app in the VM doesn’t work. The VM does start, but the app doesn’t work, just throws a 404. Why? Beats me, don’t have time to troubleshoot. Roll back a snapshot, everything works again. Conclusion, don’t update that.

          See, around here, you don’t keep your job by messing around with things that already work. They work, period, why did even feel the need to mess with that 🤨. If that’s management’s view on security, fine, so be it 🤷.

          I still file reports on things not being up to date, just so that if shit hits the fan, I’m not the one taking the fall for it.

        • 1984@lemmy.today
          link
          fedilink
          arrow-up
          3
          arrow-down
          2
          ·
          9 months ago

          Usually you upgrade everything though, not just sec patches. And it’s a risk that something stops working, and nobody wants to spend time on that…

          • 0x4E4F@lemmy.dbzer0.comOP
            link
            fedilink
            arrow-up
            2
            arrow-down
            5
            ·
            edit-2
            9 months ago

            Exactly… because it’s tidious and time consuming… and I won’t get extra pay for it. Meanwhile I’m also expected to do everything else I do… sorry, just not gonna bother at all.

        • 0x4E4F@lemmy.dbzer0.comOP
          link
          fedilink
          arrow-up
          2
          arrow-down
          1
          ·
          9 months ago

          They make multimedia, so you’ve probably “used” it at some point… or not… depends how much TV you watch.

      • ramble81@lemm.ee
        link
        fedilink
        arrow-up
        18
        arrow-down
        3
        ·
        9 months ago

        “Way too much work” — if you ever said that where I work I’d fire you or not hire you in a heartbeat. An administrator’s role is not only to the stability of the system but the security too. You’re a hackers wet dream.

        • eskimofry@lemmy.world
          link
          fedilink
          arrow-up
          12
          arrow-down
          2
          ·
          9 months ago

          The phrase “Fuck you, pay me” comes to mind.

          Cheapskates don’t get top of the line security hardening. Pay more now or suffer a breach and pay contractors $1000/hr to fix your broken shit because you paid minimum wage for an administrator position and wanted them to do 5 jobs at once.

          • 0x4E4F@lemmy.dbzer0.comOP
            link
            fedilink
            arrow-up
            10
            arrow-down
            3
            ·
            9 months ago

            This guy gets it… and probably doesn’t live in the US, cuz he knows the term “work 5 jobs at once”.

            • Miaou@jlai.lu
              link
              fedilink
              arrow-up
              2
              ·
              9 months ago

              Surely you meant the opposite? Working multiple jobs is a very USian thing. Now I’m curious, where are you from?

              • 0x4E4F@lemmy.dbzer0.comOP
                link
                fedilink
                arrow-up
                3
                arrow-down
                1
                ·
                edit-2
                9 months ago

                No no no no, I meant what I said. Working 5 jobs AT ONCE (at one postion, one place)… AT THE SAME TIME. I do hardware, software, scripting, installing, configuration, maintenance (hardware and software… the whole shbang, DB included), Linux, Windows, BSD, servers, workstations (over 800 of them, and it’s me and 2 other guys under me!)… I even do freaking rig dusting! And ALL that, for 700 freaking euros a month! Not to mention pirating, that is also included cuz… they’re too cheap to pay for licenses for… well, anything really.

                So, excuse me if I don’t care if the servers are up to date, mmmk.

                From Macedonia BTW.

                • BaskinRobbins@sh.itjust.works
                  link
                  fedilink
                  arrow-up
                  5
                  ·
                  9 months ago

                  Yeah if a company relies on some underpaid person from Macedonia to do an entire IT departments worth of crap then they deserve to get hacked lmao

                  • 0x4E4F@lemmy.dbzer0.comOP
                    link
                    fedilink
                    arrow-up
                    1
                    arrow-down
                    1
                    ·
                    edit-2
                    9 months ago

                    Told ya 🤷… you and others just have no idea how desparate things are around here, that’s why most people blame me for not updating… I believe 90% of people around here that said that, if they were in my shoes, they’d do the same 🤷.

        • 0x4E4F@lemmy.dbzer0.comOP
          link
          fedilink
          arrow-up
          8
          arrow-down
          8
          ·
          9 months ago

          One, you have no idea how much or little I’m getting paid. Two, you have no idea where I live and the struggles I have to face every day. Three, even if I do work “as expected”, I won’t get paid more (agan, you don’t know where I live).

          It’s real easy to bitch about work ethics on a full stomach while getting back from work in a nice car with heated leather seats.

          • ramble81@lemm.ee
            link
            fedilink
            arrow-up
            10
            arrow-down
            5
            ·
            9 months ago

            What I say still holds. If you ever uttered that sentiment in front of me or did not follow through on patching when asked you’d be out on your ass. Has nothing to do with your situation or what you’re making. Your righteous indignation on patching has no place in a business plain and simple.

            • thisisnotgoingwell@programming.dev
              link
              fedilink
              arrow-up
              6
              arrow-down
              1
              ·
              9 months ago

              Not trying to start an argument here but you sound very far removed from individual contributors, so maybe from your point of view it would simply look like adding it to a pile. More important than adding it to a pile is to make sure there’s systems in place to make sure OSs are patched. You wouldn’t be complaining to the IT/sysadmin guy about your servers’ vulnerability or patching schedules, you’d be talking to your cybersec department who’d have oversight. And if there’s a breach and your only defense is “I added it to the IT guys pile”, 100% you are getting fired as well.

            • InternetCitizen2@lemmy.world
              link
              fedilink
              arrow-up
              5
              arrow-down
              1
              ·
              9 months ago

              I get your point, but capitalism is about doing the least amount of work to maximize your pay. And if the owners (who have the most skin ulin the game) don’t care about infrastructure then why should anyone else?

            • 0x4E4F@lemmy.dbzer0.comOP
              link
              fedilink
              arrow-up
              6
              arrow-down
              4
              ·
              9 months ago

              No, it doesn’t… because as I said, you have your needs (food, shelter, good car, nice place to live) met… you wouldn’t be talking like this if your place was a dump or you ate the cheapes shit on the market (cuz that’s what you can afford).

          • Trainguyrom@reddthat.com
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            1
            ·
            9 months ago

            Every individual concern you’ve brought up in this thread is valid and not uncommon. The important part that @ramble81@lemm.ee is focusing on is that your mindset on how you approach these challenges is incredibly unprofessional.

            Having more to do than you realistically have time to do is the reality of working in IT. Everything the business does ultimately comes back to IT at some point in the process, so we naturally have to work with every single branch of the business.

            Being underpaid is a reality of work for most people in the modern world. The professional thing to do is to decouple your feelings of how you’re being paid from individual tasks or duties that are expected of you. “I’m not paid enough to deal with this” should be limited to tasks outside of your scope of work. If you’re ultimately not paid enough to do your job, complaining about individual tasks that are part of your job being bnove your paygrade is just saying you aren’t willing to do your job. I can’t tell you the best solution to your pay situation. Maybe changing jobs or even changing industries will help, but also changing your mindset can do wonders for your mental health. For example shifting to instead saying to yourself “I’m woefully underpaid but at least I’m working in IT and not at X” can greatly help ease the pain until you reach whatever milestone which does help improve your situation

            What you need to do regarding your workload is have a conversation with your manager/superior about prioritization. You say “hey, I have this this this and this that need to get done right now and I can’t realistically do them all, what do you want me to prioritize and deprioritize” and if something important hasn’t been given priority in a long time (such as patches) you need to then push back and say “we haven’t been able to apply security patches in quite a while, I think we should reprioritize this so we can put some time into patching this week” this is how you manage a gigantic workload is by shifting priorities. The longer important maintenance tasks are ignored, the larger the impact and the harder it will be to complete the tasks

            • 0x4E4F@lemmy.dbzer0.comOP
              link
              fedilink
              arrow-up
              2
              arrow-down
              1
              ·
              edit-2
              9 months ago

              The important part that @ramble81@lemm.ee is focusing on is that your mindset on how you approach these challenges is incredibly unprofessional.

              I know it is.

              Now, ask me why that is.

              The professional thing to do is to decouple your feelings of how you’re being paid from individual tasks or duties that are expected of you. “I’m not paid enough to deal with this” should be limited to tasks outside of your scope of work.

              Lol 😂, no one has actually told me to keep the servers up to date, the only thing I was ever told was “keep shit running”. I’ve done updates on my own incentive, since I’m the senior IT engineer in the company. When things turn to shit after an update, hey, I’m rolling back a snapshot, I did not sign up for this 🙌.

              And that is basically it 🤷… I get the same salary regardless if I do them or not. After a few failed ones, I just gave up. F it, not worth the time or the effort.

              If you’re ultimately not paid enough to do your job, complaining about individual tasks that are part of your job being bnove your paygrade is just saying you aren’t willing to do your job.

              Nope, I’m saying “Fuck you, pay me!”.

              You think they care about updates and security? I’ve mentioned it a few times at meetings… “yeah, we’ll talk about that later”. OK 🤷. They obviously have no idea how fucked up things can get if you’re not up to date regarding security… but hey, they have been warned more than a few times 🤷.

              What you need to do regarding your workload is have a conversation with your manager/superior about prioritization. You say “hey, I have this this this and this that need to get done right now and I can’t realistically do them all, what do you want me to prioritize and deprioritize” and if something important hasn’t been given priority in a long time (such as patches) you need to then push back and say “we haven’t been able to apply security patches in quite a while, I think we should reprioritize this so we can put some time into patching this week” this is how you manage a gigantic workload is by shifting priorities. The longer important maintenance tasks are ignored, the larger the impact and the harder it will be to complete the tasks

              🤦… dude, you really have no idea where I live 😂🤣😂… otherwise you wouldn’t be saying this.

              Things have been said more than once… I have asked, have pleeded for more personel… deaf ears. I have put it in writing, no use. Fine, then I just keep things running and that’s basically it 🤷.

              Oh, and regarding workload, I already prioritize. The priority is to keep shit running, not to be up to date (obviously)… so, I just keep things running.

          • Miaou@jlai.lu
            link
            fedilink
            arrow-up
            1
            arrow-down
            2
            ·
            9 months ago

            Well, getting promoted is difficult when you watch Netflix or browse lemmy on company time

            • 0x4E4F@lemmy.dbzer0.comOP
              link
              fedilink
              arrow-up
              3
              arrow-down
              1
              ·
              edit-2
              9 months ago

              No no no, I am not getting promoted anyway… no promotions here. No one ever gets promoted, you don’t get bonuses, you don’t get overhours pay, you don’t get raise… so why bother 🤷.

      • bushvin@lemmy.world
        link
        fedilink
        arrow-up
        2
        ·
        9 months ago

        The I can only recommend you to start automating everything you do, to make tour job easier and make more time to slack 😝

        Start small, and build on that.

        Try Ansible, it is easy and allows you to do some really cool stuff. It helped me migrate 500+ systems from KVM to vmware, where no commercial tool was able to help me…

        • 0x4E4F@lemmy.dbzer0.comOP
          link
          fedilink
          arrow-up
          1
          arrow-down
          1
          ·
          edit-2
          9 months ago

          Creating automation takes time. I have done it with certain things, but not everything. Some things are just way too complicated to easily create automation for them.

          Though I do agree that things like Ansibel can help and I have been thinking about this for a while, but then I’d have to drop everything else I’m doing for like at least 2 weeks and do JUST that, nothing else… which is not an option currently.

          Plus, there is always the risk of “shit not working” after an update… and frankly, we’ve had that before and again, I am not getting paid enough to deal with the backlash of things like that happeneing.