How do I filter through list items based on multiple data attributes with jQuery?

Here is my HTML with all of my filters:

  <div class="search-filters">
    <div class="filter-block">
      <label for="employee_type">Employee Type</label>
      <select id="employee_type" class="category">
        <option value="0">All</option>
        <option value="designer">Designer</option>
        <option value="manager">Manager</option>
        <option value="developer">Developer</option>
        <option value="quality-assurance">Quality Assurance</option>
      </select>
    </div>
    <div class="filter-block">
      <label for="years_xp">Years Experience</label>
      <select id="years_xp" class="category">
        <option value="0">Select years Experience</option>
        <option value="1">1 year</option>
        <option value="2">2 years</option>
        <option value="3">3 years</option>
      </select>
    </div>

  </div>

And my result list is below:

 <ul class="result-set">
   <li class="result" data-employee-type="designer" data-xp="3" >John Smith</li>
   <li class="result" data-employee-type="manager" data-xp="2" >Billy Joe</li>
   <li class="result" data-employee-type="developer" data-xp="5" >John Appleseed</li>
 </ul>

So what I'm trying to do is filter the results with jQuery using the data attributes and select drop-downs on change. Right now only one or the other filter works with my jQuery code below. I tried using the .filter function in Jquery but it still only works when one drop-down is changed. If I change both it seems to be ignoring one condition.

My jQuery:

var category_filters = [];

$('.category').on('change', function() {
    category_filters = [];

    $('.search-filters select').each(function() {
        if ($(this).val() != 0) {
            var category = $(this).val();
            category_filters[$(this).attr('id')] = category;
        }
    });

    var employee_type = "";
    var years_xp = "";

    if ('employee_type' in category_filters) {
        employee_type = category_filters['employee_type'];
    }
    if ('years_xp' in category_filters) {
        years_xp = category_filters['years_xp'];
    }

    $(".result-set .result").hide().filter(function() {
        if (employee_type != "") {
            if ($(this).attr('data-employee-type') == employee_type) {
                return true;
            }
        }
        if (years_xp != "") {
            if ($(this).attr('data-xp') == years_xp) {
                return true;
            }
        }
    }).show();

});

Any ideas on what i'm doing wrong? I've been stuck on this for almost 6 hours now and have looked at other examples but they just don't seem to work with my case.